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EXECUTIVE SUMMARY 
Texas Study of Students at Risk 


The Texas Study of Students at Risk (TxSSAR) is a comprehensive evaluation examining the 
effectiveness of three state-level programs with the common goal of helping students at risk of failure to 
achieve academically. The study comprises investigations of the Optional Extended Year Program 
(OEYP), the Texas After School Initiative (TASI), and the Ninth Grade Success Initiative (NGSI), and 
case studies of districts that received NGSI grants. The evaluation covers a four-year period between the 
1999-2000 and 2002-03 school years. 

OPTIONAL EXTENDED YEAR PROGRAM 

Eirst established by the 73rd Texas Eegislature in 1993, the Optional Extended Year Program (OEYP) is a 
state-funded program with the goal of meeting the needs of elementary and middle school students 
(kindergarten through grade 8) who are at risk of not being promoted to the next grade level. Non- 
competitive grants allow districts to provide an extended-year program for up to 30 instructional days for 
eligible students, with the ultimate goal of reducing retention rates. 

PROGRAM ELEMENTS 

Characteristics of districts. Between 1999-2000 and 2002-03, the total number of districts receiving 
OEYP funds was 695, 682, 672, and 684, respectively. The average award actually paid to districts was 
between $76,000 and $80,000. Paid awards ranged from $317 to over $5,000,000. 

Characteristics of students. About 190,000 students participated in the OEYP each year. Participants 
are distributed across grades 1 through 8, with the largest proportion being third graders. Compared to the 
state, OEYP served a greater proportion of Hispanic students (about 64%), slightly more African 
American students (about 18%), and substantially less White students (about 17%). OEYP students were 
also more likely to be economically disadvantaged (about 79%) and limited English proficient (about 
31%). 

Program types. About two-thirds of OEYP students participated in an extended-year or intercession 
program only, whereas about one-fourth only participated in an extended-day program. Across four 
school years, the prevalence of extended-day and extended-week programs increased, while the emphasis 
on extended-year or intersession programs decreased. 

Program activities. OEYP instructional activities focused most often on reading/language arts and 
mathematics. Districts mainly focused their professional development opportunities for teachers and staff 
on instructional strategies and strategies for teaching students at risk. Districts most frequently planned to 
involve parents through conferences, parent workshops, and various communication strategies. On 
average, the number of OEYP instructional days available for students declined across four years (from 
20 to 15). Intercessions or extended-year programs had the largest number of instructional days (between 
19 and 21 days each year). 

EFFECT ON STUDENTS 

To determine the effectiveness of the OEYP program, we examined OEYP students’ attendance and 
retention rates and performance on state-level assessments. 

Attendance. Students’ average OEYP attendance rates for four school years (81% to 86%) are 
considerably lower than their attendance rates during the regular school year (about 96%). Students 
attended OEYP extended-year and intercession programs at a higher rate (from 86% to 90% of 
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instructional days) than extended-day (54% to 80% of days) or extended-week (62% to 70% of days) 
programs. 

Retention. Districts are using student retention in the early grades as a means to support academic 
performance. Across four years, about 23% of OEYP first graders, 16% of second graders, and 9% of 
third graders were retained. In contrast, retention rates for students in grades 4 through 8 were typically 
less than 5%. Compared to state averages, retention rates for OEYP students in grades 1 to 3 are far 
higher (about 17, 12, and 6 percentage points, respectively) but only slightly higher for students in grades 
4 to 8 (about 2 percentage points). Eor all grade levels, student retention rates tended to increase across 
the four OEYP program years. 

State-level assessments. Passing rates on state assessments (TAAS reading, math, writing, science, 
social studies, and all tests) were well below state averages for the four OEYP student cohorts studied. 

Eor cohort 1 (1999-2000) and cohort 2 (2000-01) students, TAAS passing rate gains (from the year before 
to the year after full OEYP participation) exceeded state gains. However, the TAAS to TAKS passing rate 
gains for cohort 3 students (2001-02) were mostly less than state gains. Thus, the achievement gap 
between OEYP students and state averages was narrowed for cohorts 1 and 2, but not for cohort 3. 

ASSOCIATION BETWEEN PROGRAM ELEMENTS AND STUDENT OUTCOMES 

To further explore the association between OEYP student and district characteristics and TAAS reading 
and mathematics TEI scores, researchers used hierarchical linear modeling (HEM). Separate analyses 
were conducted using participants in 1999-00 (cohort 1) and 2000-01 (cohort 2). Analyses were also 
conducted for retention. 

Program type. Controlling for important student characteristics (i.e., academic and social background), 
extended-day participants had higher TAAS reading and mathematics scores than extended-year/ 
intercession participants. Thus, students receiving assistance during the school year may do better 
academically than those who attend an intercession or summer school after failure. 

Instructional days. There was no positive relationship between the instructional days students spend in 
OEYP (up to 30 instructional days) and TAAS scores. However, for otherwise similar students, more 
instructional days in OEYP decreased the chances of retention for extended-year/intercession participants. 
This may reflect the fact that successful completion of a fixed number of scheduled days of instruction for 
extended-year/intercession programs precludes retention. In contrast, for extended-day participants, fewer 
instructional days in OEYP decreased chances of retention. Eindings on extended day suggest that 
students may receive assistance in extended-day programs on an as-needed basis, and students with less 
need may attend fewer days. 

Attendance. A student’s school attendance rate was an important predictor of academic performance. 
School attendance had a stronger influence on TAAS mathematics scores than on TAAS reading scores. 

In addition, for otherwise similar students, an increase in the school attendance rate decreased the chances 
of retention. 

District. After controlling for student-level characteristics, OEYP students’ academic achievement and 
chance of retention varied significantly by district. This suggests that some districts and schools are more 
successful in meeting the needs of students in at-risk situations. 

Context. Evidence confirms the importance of the school context. Other student-level factors being equal 
and net of district social context and OEYP expenditures, OEYP students having higher achieving 
classmates performed better in TAAS reading and mathematics. 

Per-pupll expenditure. There was no significant relationship between OEYP dollars spent per pupil and 
TAAS reading and mathematics scores. Moreover, higher OEYP per-pupil expenditures were associated 
with a slightly increased chance of student retention. Results suggest that how districts use available 
resources is critically important in improving outcomes for students at risk. 
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IMPLICATIONS FOR ADDRESSING STUDENTS’ NEEDS 


Enhancing the academic prospects of at-risk students hinges on overaii improvement of iearning 
opportunities in schoois and ciassrooms. Findings reinforce the importance of improving the overall 
school environment as a means to enhance the learning opportunities of students at risk. Results for 
“value-added” modeling suggest that some districts and schools are more successful than others in 
supporting the academic performance of students at risk. Results for this study are consistent with other 
research citing the importance of the school context (Stringfield & Datnow, 2002; Bitting, Cordero, & 
Baptiste, 1992; Waxman, 1992). 

Efforts directed at improving student attendance during the regular school year may have a 
greater effect on student achievement than remedial interventions. Results reinforce the importance 
of school attendance in the academic success of students in at-risk environments. School attendance was 
an important predictor of performance on state-level assessments, especially mathematics, and attendance 
was also associated with decreased chances of retention. 

Low student attendance in extended-day, -week, and -year programs limits program effectiveness. 

Findings for four student cohorts suggest that student attendance in OEYP programs was sporadic 
(ranging from 54% to 90% of instructional days). Moreover, the number of available OEYP instructional 
days declined from 20 to 15 across four grant years. Thus, it is doubtful that the number of days available 
and attended is adequate to substantially impact either achievement or retention (e.g.. Glass, 2002). 

Little is known about the quality of programs funded by OEYP. A review of district proposals 
revealed that OEYP programs focus primarily on reading and mathematics and many districts use 
computer-assisted programs to deliver instruction (usually learning systems for basic skill acquisition). 
Beyond this, there is little available evidence on program quality. 

Student retention rates increased across four years, especially for first, second, and third graders. 

OEYP was unsuccessful in achieving its primary goal — the reduction of student retention. Retention rates 
for students at risk increased across four years as districts increasingly retained students in first, second, 
and third grade. Retention also increased slightly for grades 4 to 8 students (about 1 percentage point). 
Increased retention of at-risk students is troubling in light other studies showing detrimental effects on 
students (e.g., Nagaoka & Roderick, 2004). 

The cost-effectiveness of the OEYP is questionable. Associations between OEYP funding levels and 
both student achievement and retention suggest there was no significant relationship between OEYP 
dollars spent per pupil and academic achievement or reduced retention. Eindings raise questions about the 
cost-effectiveness of the initiative statewide. 

State-level initiatives aimed at improving instruction and learning for students at risk should be 
accompanied by evaluations to study program effectiveness. Conducting scientifically rigorous 
evaluations of statewide initiatives relies on designing and conducting studies at the onset of funding and 
program implementation. Eunding for future initiatives supporting students at risk should be accompanied 
by resources for program evaluations. 
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TEXAS AFTER SCHOOL INITIATIVE 


In 1999, the Texas Legislature created the Texas After-School Initiative. The program funds after-school 
programs targeting middle school students, ages 10 to 14, who are at risk of academic failure and/or at- 
risk of committing juvenile offenses. This study includes 60 districts and 194 campuses receiving both 
original and continuation TASI funding. 

PROGRAM ELEMENTS 

Characteristics of districts. TASI programs were more heavily concentrated in larger districts (more 
than 10,000 students). Smaller districts seldom had TASI programs. 

Characteristics of students. During four program years, nearly 102,000 unique students participated in 
the program (based on an estimated count). TASI programs increased from about 13,000 students (spring 
2000) to almost 32,000 (2002-03). Participants included a nearly equal proportion of sixth, seventh, and 
eighth graders. More than three-fourths of students were Hispanic (about 53%) and African American 
(22%), and about two-thirds were economically disadvantaged (59%). Approximately 8% of students had 
repeated one grade, and about 1% had multiple retentions. 

Program characteristics. TASI programs addressed student needs by incorporating three components: 
an academic-based curriculum linked to state standards, a character/citizenship education component, and 
a plan for parental and/or mentor involvement. Programs typically offered about four instructional days 
per week, with nearly two program hours each day. The mean number of instructional days varied across 
years from about 49 to 58. Districts dedicated the greatest percentage of after-school time to the 
academic-based component (more than 40% of time for three-quarters of programs). 

Academic component. Instructional technology was commonly used in the academic component. 
Lightspan was the most frequently used program and general use of technology in academic activities 
(such as word processing and Internet) was also prevalent. Tutoring was a commonly used strategy 
throughout the grant period. 

Character education/citizenship component. Districts used a combination of commercial programs 
{Voyager, ROPES, Character Counts, etc.), external supports (guest speakers, field trips, community 
service, etc.), and other activities (athletics, fine arts, etc.) as part of the TASI character 
education/citizenship component. 

Parent and mentor component. Traditional means of communicating with parents (meetings, mail, 
telephone) and involving parents (training, volunteering) were most commonly used in TASI programs. 
Mentors most often served as tutors or guest speakers. 

EFFECT ON STUDENTS 

Core-subject course passing rates. The majority of TASI students (84% to 89%) passed core content- 
area courses. Course passing rate were slightly lower for mathematics (84% to 86%). Students meeting 
state Compensatory Education requirements or having other risk factors had lower passing rates (up to 10 
percentage points less than more advantaged peers). 

Discipline referrals. Nearly one-fifth of TASI students (17% to 20%) had four or more office referrals. A 
small percentage of students were referred to alternative education programs (about 5%) or juvenile 
justice programs (less than 1%). Since disciplinary referrals remained relatively stable across program 
years, there appeared to be no association between TASI and the improvement of student discipline. 

Attendance rates. In general, students’ attendance rates did not improve over time. However, attendance 
rates for TASI students during the program year (about 96%) were consistently higher than rates for a 
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comparison group of non-TASI students (about 95%). TASI students who are in grade for the first time 
have higher attendance rates than students repeating a grade level. Although attendance rates for students 
repeating a grade level declined over time, a slightly positive change was observed during the program 
implementation year for three cohorts. 

State-level assessments. TASI students had lower TAAS passing rates for both reading and 
mathematics compared to non-TASI students, but the achievement gap between groups narrowed slightly 
for three student cohorts. Despite apparent progress, the achievement gap between TASI and non-TASI 
students increased in both reading and mathematics for students who completed the TAKS assessments 
(cohort 4). For the small number of students repeating their grade level, the passing rate gap on state-level 
assessments was narrowed between TASI and non-TASI student cohorts. 

Retention. Retention rates for TASI students declined across cohorts (3% to 2.2%), and in cohort 3, 

TASI students had slightly lower retention rates than a comparison group of non-TASI students (2.2% 
compared to 2.5%). For the small number of students repeating their grade level, across-cohort trends 
showed that TASI students had slightly lower retention rates (3%, 2.9%, 2.2%) than non-TASI students 
over time (2.5%, 3%, 2.5%). 

ASSOCIATION BETWEEN PROGRAM ELEMENTS AND OUTCOMES 

Researchers used hierarchical linear modeling (HLM) to further explore the association between TASI 
student and district characteristics and academic achievement. Analyses involved participants in 2000-01 
(cohort 2) and 2001-02 (cohort 3). Separate analyses were also conducted for retention. 

Instructional days. After controlling for the effect of student characteristics (academic and social 
background), there was no positive relationship between the number of instructional days students spend 
in TASI (up to 189 days) and TAAS scores. Thus, the academic component was not optimally effective in 
improving student academic performance. In contrast, more instructional days in TASI were associated 
with a marginally decreased chance of retention for cohort 2 students (2001-02). In general, TASI appears 
to have had little or no impact on achievement but may have been somewhat effective in reducing student 
retention. 

Attendance. A student’s school attendance rate was a significant predictor of academic performance. 
Higher school attendance rates were associated with higher TAAS reading and mathematics scores. In 
addition, for otherwise similar students, an increase in a student’s school attendance rate decreased the 
chances of retention. 

Per-pupil expenditures. Consistent with findings for the OEYP, there was no significant relationship 
between TASI dollars per pupil and TAAS reading and mathematics scores. Likewise, there was no 
significant relationship between dollars per pupil and retention rates. 

IMPLICATIONS FOR ADDRESSING STUDENTS’ NEEDS 

After-school programs, as they are currently designed, appear only marginally successful in 
improving the academic performance of the majority of student participants. For students in their 
grade for the first time (the majority of TASI participants), program participation had no discemable 
relationship to improved school attendance rates and only a modest correlation with increased TAAS 
scores. Despite some TAAS gains in reading and mathematics, the majority of TASI students lost ground 
compared to their non-TASI counterparts on the TAKS assessments, especially in mathematics. 

After-school programs may provide the greatest benefit for students who have been retained in 
grade. There was a stronger relationship between TASI participation and both attendance and TAAS 
scores for students repeating a grade level. A slightly positive change was observed for student attendance 
during the TASI program year. Moreover, for a small number of students repeating their grade level, the 
passing rate gap on state assessments was narrowed between TASI and non-TASI students. 



Reducing student retention through participation in an after-schooi program does not necessariiy 
transiate into improved academic achievement. Results show that retention rates were slightly 
reduced for TASI participants across all categories of students. However, simply preventing student 
retention did not ensure increased knowledge and skills as measured hy state assessments. 

The cost-effectiveness of after-schooi programs shouid be examined more comprehensiveiy. The 

effectiveness of after-school programs, and especially cost-effectiveness, remains uncertain. Some trends 
have been revealed, hut a more in-depth examination of specific programs is needed in order to 
understand what programs work, for whom, and under what circumstances. 

NINTH GRADE SUCCESS INITIATIVE 

From 1999 to 2002, the state appropriated a total of $170 million for the Ninth Grade Success Initiative 
(NGSI) to support school districts’ efforts to help ninth graders stay in school and succeed academically. 
The goal was to increase graduation rates in Texas public schools by reducing the number of students 
who were retained in or dropped out of the ninth grade. Funds went toward expanding or enhancing 
existing programs, or creating new programs to increase academic performance and attendance rates and 
reduce dropout rates for ninth graders who had not earned — or were unlikely to earn — sufficient credit to 
advance to tenth grade or eighth graders who were promoted but considered academically at risk. This 
study involved 226 school districts receiving both original and continuation funding. 

PROGRAM ELEMENTS 

Characteristics of districts. NGSI district size distributions differ from the state overall, with more than 
half of grant recipients (58%) either midsize to very-large districts (3,001 to more than 25,000 students). 

Characteristics of students. During four program years, nearly 390,000 unique students participated in 
the NGSI (based on an estimated count). Participants increased from 32,535 students (spring 2000) to 
106,325 (2002-03). Substantially fewer students participated during summer terms, but enrollments 
climbed steadily during the grant period (from 19,508 to 31,607). 

The majority of NGSI students (80% or more during regular terms) were ninth-grade students at-risk of 
not earning sufficient credits to advance to tenth grade. About three-fourths of students were either 
Hispanic (56%) or African American (17% during regular terms and 20% during summer). The majority 
of students were in the ninth grade for the first time (80% or more). Percentages of newly promoted ninth 
graders served in NGSI declined across summer terms (from 33% to 9%). 

Program characteristics. NGSI programs typically used several activities to serve at-risk ninth graders. 
Tutoring, instructional technology, individual instruction, group instruction, and counseling were reported 
most often. During the regular school year, students accrued course credit primarily through regular 
classroom instruction, but repeat ninth graders were more likely to accrue credit through computer-aided 
instruction (e.g., PLATO or NovaNET self-paced learning systems). 

EFFECT ON STUDENTS 

Core-content courses. Passing rates for core subject-area courses remained relatively stable across 
NGSI grant terms, with about 70% of students passing Algebra I during regular terms and about three- 
fourths or more of students passing Biology, Integrated Physics and Chemistry (IPG), World Geography, 
and English I. Course passing rates increased for summer terms (about 80% to 95% passing) but student 
enrollments decreased substantially. Students in ninth grade for the first time and newly promoted ninth- 
grade students had higher passing rates for core courses than students who did not earn sufficient credits 
for promotion. 

Attendance. In general, NGSI students’ attendance rates did not improve across grant terms. Eor both 
NGSI and a comparison group of non-NGSI students, first-time ninth graders had substantially higher 
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attendance rates (about 92% to 96%) than repeat ninth graders (about 83% to 93%). NGSI first-time ninth 
graders had slightly lower attendance rates than their non-NGSI peers (about 0.5 to 2.0 percentage 
points). Attendance rates for repeat NGSI ninth graders, however, were typically near or surpassed non- 
NGSI comparison groups. Attendance rates for both first-time and repeat NGSI students declined across 
time. 

State-level assessments. NGSI students had lower TAAS passing rates for both reading and math 
compared to non-NGSI students, but the achievement gap between groups narrowed (to 3.7 points in 
reading and 6.6 points in math). Despite encouraging results for TAAS, the achievement gap widened 
substantially for students in cohorts 3 and 4 who completed the TAKS (to about 18 percentage points for 
math). NGSI repeat ninth graders had similar passing rates on state assessments compared to non-NGSI 
students for both reading and mathematics. However, for both student groups, passing rates declined 
substantially for TAKS reading and math. 

Retention rates. Although NGSI student retention rates remain high (21.8% in 2002-03), evidence for 
four program years reveals that NGSI retention rates have decreased more than rates for non-NGSI 
students (-7.7 points compared to -3.3). First-time ninth graders had greater declines in retention rates 
than non-NGSI students. Hispanic and African American students had the highest retention rates (25% in 
2002-03), but both groups had the greatest reductions in retentions across program years (-7.7 and -9.4 
points, respectively). Retention rate declines were similar for economically disadvantaged and advantaged 
students. 

ASSOCIATION BETWEEN PROGRAM ELEMENTS AND OUTCOMES 

Researchers used hierarchical linear modeling (HLM) to further explore the association between NGSI 
student and district characteristics and academic achievement. Analyses involved NGSI participants in 
2000-01 (cohort 2) and 2001-02 (cohort 3). Separate analyses were also conducted for retention. 

Instructional days. After controlling for the effect of student characteristics (academic and social 
background), there was no positive relationship between the number of days students participated in 
NGSI and achievement scores on TAAS/TAKS reading/ELA or mathematics assessments. In fact, a 
negative relationship existed between days and TAAS achievement. In contrast, more instructional days 
in NGSI were associated with a slightly decreased probability of retention for student in both cohorts. 

Attendance. Ninth graders’ school attendance was an important predictor of academic performance. For 
otherwise similar students, a student’s school attendance rate was positively associated with both 
TAAS/TAKS reading/ELA and mathematics achievement. Moreover, for both student cohorts an increase 
in a student’ s school attendance rate was associated with a decreased chance of retention. 

District. After controlling for student-level characteristics, NGSI students’ academic achievement varied 
significantly by district. This suggests that some districts and schools were more successful in meeting the 
needs of ninth graders. 

School context. Eor students in cohort 2, having higher achieving classmates was associated with a 
slightly reduced chance of retention. There was also a slightly positive association between the number of 
NGSI program days offered students and TAAS reading and math scores. Thus, districts that made a 
larger number of days available for the program had greater success in improving student achievement. 

Per-pupil expenditure. Similar to findings for other initiatives, there was no significant relationship 
between NGSI dollars per pupil and student achievement. However, for NGSI, higher per-pupil 
expenditures were associated with reduced student retention. 

IMPLICATIONS FOR MEETING STUDENTS’ NEEDS 

Few districts designed programs for newly promoted ninth graders who lacked minimum skills for 
successful course completion. The majority of students served by NGSI programs were in the ninth 
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grade for the first time (80% or more eaeh term). The pereentages of newly promoted ninth graders served 
in NGSI declined across summer terms (from about 33% to 9%). 

African American students were more iikeiy to be enroiied in summer schooi programs, which 
typicaiiy heiped students to recover credits for faiied courses, rather than program interventions 
during reguiar schooi terms. The percentage of African American students in NGSI programs increased 
during summer terms — thus, interventions for those students focused more often on remediation of 
academic failure rather than proactive efforts to improve success. 

Evidence from district NGSi reports shed iittie iight on the identification of effective programs for 
students at risk. The instructional and learning focus of NGSI programs remains unclear because the 
majority of districts used multiple approaches. It is impossible to determine the effectiveness of a program 
when students apparently receive multiple interventions. 

Students repeating ninth-grade coursework were more iikeiy to accrue credit through seif-paced 
computer-aided instruction. However, there is iittie evidence to support program effectiveness. 

Districts invested a substantial proportion of NGSI grant resources in self-paced instructional systems 
(e.g., PLATO or NovaNET) in labs for tutorials, credit recovery, or credit accrual for repeat ninth graders. 
Although most educators view self-paced programs positively, little empirical evidence is available on the 
effectiveness of programs in addressing the needs of students at risk. 

Improved performance in core-subject area coursework is critical to the success of at-risk 
students. NGSI had no discernable effect on ninth graders’ course passing rates for Algebra I, Biology, 
IPG, World Geography, or English I. About 70% or less of students passed Algebra I each year and about 
three-fourths of ninth graders passed other courses. Algebra I is a major obstacle for many students, with 
nearly 30% of first-time and 40% of repeat ninth graders failing algebra. 

Poor school attendance Jeopardizes the academic success of students at risk. Although improving 
student attendance was a key goal, results show that NGSI had no positive, sustained effect on ninth 
graders’ school attendance. More importantly, attendance rates for both first-time and repeat ninth 
graders decline as they progress to higher grade levels. Eindings are particularly important because school 
attendance emerges as a significant predictor of student academic achievement and reduced chances of 
retention. 

Modest accomplishments for the NGSI suggest that the $170 million invested in the initiative did 
not achieve program goals for students at risk. Eindings suggest that, as a whole, the NGSI program 
was somewhat effective in reducing ninth-grade retention rates but fell short of accomplishing other 
important goals such as improved attendance and increased academic achievement on state assessments. 
Moreover, analyses revealed no significant associations between per-pupil expenditures and the academic 
achievement of students. Per-pupil expenditures were associated with a slightly reduced probability of 
student retention, but the effect may not justify the cost. 

CASE STUDIES OF NGSI GRANTS 

Researchers conducted case studies of NGSI grants to gain a greater understanding of issues facing large 
numbers of at-risk students, many of whom, despite potentially receiving services as early as 
kindergarten, still reach ninth grade unprepared to succeed academically in high school. Case studies of 
1 1 districts focused on NGSI programs and the broader high school contexts in which they operated. 

PROGRAM ELEMENTS 

Programs for Newly Promoted Ninth Graders 

Pew districts offered programs for newly promoted ninth graders who lacked minimum skills for 
successful course completion. Educators in districts that offered programs believed newly promoted ninth 
graders who participated in summer programs benefited from reduced class size, active learning, bonding 


viii 



with teachers, and high school orientation. Although educators viewed programs as worthwhile and 
effective, few students participated and most programs were discontinued. 

Programs for First-Time and Repeat Ninth Graders 

Districts invested the hulk of NGSI resources in services for ninth graders who were at-risk of not earning 
sufficient credit or had not earned sufficient credit to advance to grade 10. 

Computer-assisted instruction. Most districts invested a substantial proportion of grant funds in 
technology for computer-assisted instruction. Instructional technology most frequently included 
comprehensive programs supporting self-paced credit recovery or skill remediation (e.g., PLATO, 
NovaNET). A few districts purchased programs for comprehensive coursework or supplemental 
instruction. 

• Self-paced credit recovery labs. Staffing of self-paced credit recovery lahs for at-risk students most 
often involved one certified teacher who managed coursework in several core-suhject areas. One very 
large district took a more comprehensive approach hy establishing Learning Labs with computer- and 
text-based assignments, instructional support, and social services. Almost all educators and students 
believed self-paced courseware benefited students by offering alternative means for credit recovery, but 
learning outcomes for comprehensive services were most promising. Concerns with self-paced learning 
programs include software quality, TEKS and TAKS alignment, student attendance, recruitment of 
effective teachers, and whether earned credits reflect content mastery. 

• Computer-assisted algebra coursework. Two districts implemented comprehensive algebra 
coursework. Most educators viewed I CAN Learn (a lab-based computerized algebra curriculum) and 
Cognitive Tutor (a combination of computer- and text-based assignments) positively, believing they 
helped ensure curricular consistency and improved student algebra performance. 

• Supplemental computer-assisted instruction. Computer-assisted instruction in English and math 
labs appeared to improve learning for some students through clear directions, examples, and help with 
understanding the basics. Limited access to supplemental instruction and uneven program 
implementation, however, diminish the potential impact on student achievement. 

Extended-day programs. A few districts funded extended-day programs with tutorials or credit recovery 
opportunities for ninth graders. Students who took advantage of extended-day tutorials apparently 
benefited, but student participation was a major obstacle. Most students at risk are unlikely to attend 
extended-day tutorials voluntarily. Examples of successful programs were rare, but better participation 
was associated with programs that were well organized and scheduled, obtained parent consent and 
support, used alternative instructional approaches (e.g., computer-assisted learning), and provided 
transportation. 

Extended-year programs (summer school). Nearly all districts used NCSI funds to provide credit 
recovery opportunities for ninth graders through summer programs. Programs varied by duration, daily 
schedule, earnable credits, course delivery method, and core-subject availability. Summer programs 
reportedly allowed some students to recover credits, avoid retention, and remain with their peers in tenth 
grade. Districts face challenges in getting ninth graders to attend summer school, ensuring regular 
attendance, setting high expectations for student work and behavior, and helping students prepare for 
subsequent coursework. 

Whole-school improvement. Districts seldom used NGSI grants to transform their high schools’ 
approach to serving students at risk. However, a few undertook organizational restructuring, invested in 
course improvement, or provided teacher professional development. 

• School-within-a-school. Two districts used schools- within-a school to create smaller and more 
supportive environments in high schools. Ninth-grade teams reportedly strengthened student and 
teacher support, improved parent communication, increased focus on student progress, and reduced 
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retention. Some educators believe ninth graders are carrying forward organizational habits and 
responsible behaviors developed in the school-within-a-school. 

• Enhancement of core-subject courses and professional development Core-subject course 
enhancement occurred infrequently through NGSI grants. Educators in two districts used computer- 
assisted instruction to enhance Algebra I coursework for ninth graders. Similarly, professional 
development was used in only a few districts as a means to improve teaching and learning in core- 
subject area classrooms. 

EFFECT ON STUDENTS 

Research design and confounding factors make causal inferences about NGSI effects on the case-study 
districts impossible; however, data trends across the grant period reveal some increases in student 
attendance, decreases in retention rates, and improved algebra performance. Despite improvements, 
student attendance rates are generally less than 95% (No Child Left Behind test-participation standard), 
nearly one-fifth of ninth graders are not promoted, and fewer than half of ninth graders typically passed 
end-of-course algebra exams. 

SCHOOL CONTEXT AND EDUCATIONAL ENVIRONMENT 

Each grant program operates within the broader campus and school district as a whole — therefore, to 
better understand student performance, researchers examined not only the NGSI program but also the 
school context experienced by ninth graders at risk of failure. 

Standards and expectations. In nearly all high schools visited, the Recommended High School 
Program is currently the default curriculum. Many districts have established more rigorous promotion 
standards to ensure that ninth graders are prepared for TAKS, and some high schools have toughened 
student promotion standards. Many high schools now require students to complete six credits rather than 
five to advance to tenth grade, and some require students to complete core-subject area courses as well. 

Structure and organization. Although most high schools retain the traditional grades 9-12 structure, 
some have created smaller, more supportive units within the high school. Scheduling approaches vary 
widely, but high schools appear to be shifting from block schedules (90-minute periods) to traditional, 
single-period schedules (50-minute periods). A few high schools modified their schedules to give 
extended learning time to ninth graders considered at risk of academic failure, primarily in algebra and 
English. Two districts created ninth-grade schools with students housed in a separate building near an 
affiliated senior high school. This configuration reportedly benefits ninth graders by easing crowding 
(about 800-900 students per school), reducing discipline problems, and creating an environment that 
allows maximum attention to students’ academic and emotional needs 

Extra academic assistance. Ah high schools visited offer extra academic assistance to students 
considered at risk, but some take a more structured approach. Academic assistance frequently helps 
students prepare for the state assessment (TAKS), complete assignments, or make-up assignments or 
excessive absences. Although educators and student participants believe tutorials are helpful, most at-risk 
students do not attend unless they are required. Barriers to participation in tutorials include transportation 
issues, lack of motivation, scheduling difficulties, after-school conflicts, and perceived benefits. 

Guidance and counseling. Guidance and counseling services for students in at-risk situations are 
limited in many high schools by counselor-to-student ratios that exceed recommended standards. Contacts 
between at-risk ninth graders’ and counselors are limited primarily to the selection of courses or 
programs; older students are more likely to receive information about jobs and careers, or how to improve 
academic work. Ninth graders’ interactions with counselors on high school plans occur most often in 
groups rather than individually. 
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Teachers and teaching. Ninth-grade teachers are fairly experienced, hut a substantial proportion (about 
40%) comes to teaching through non-traditional certification. Educators raise concerns about the 
assignment of new and inexperienced teachers to ninth-grade courses. 

• Perceptions of effective instruction. Beliefs about teaching practices vary widely among high 
school teachers, with some advocating learner-centered approaches and others favoring traditional 
methods. Students who are at risk say good teachers provide clear explanations, encourage active and 
meaningful learning, make class interesting, establish personal relationships, use small-group activities, 
and offer individual help. Both teachers and students advocate active and meaningful learning 
experiences, varied (or interesting) instructional approaches, and positive interpersonal relationships. 

• Teachers’ ciassroom practices. Teachers expressed opinions on effective instruction, as cited 
above, differ from observed practice. High school classrooms are organized most often for whole-class 
instruction. Students seldom work collaboratively with peers. Teachers spend the greatest proportion of 
class time providing whole-group instruction and monitoring students as they work independently on 
assignments. Teachers seldom ask mentally challenging questions or questions that help at-risk 
students see the relevance of subject matter to their lives. Since teachers have little access to 
technology in classrooms, it is seldom used to support instruction and learning. 

Students and learning. Students considered at risk spend the greatest part of their time listening to 
teacher presentations or independently completing short-answer activities or worksheets. Most class 
discussions are teacher controlled question and answer exchanges. Overall, observed practices raise 
questions about teachers’ understanding of students as learners, especially research-based conceptions 
(e.g., Bransford, Brown, & Cocking, 2002). 

• Disengagement from high schooi and iearning. Evidence from various sources points to at-risk 
students’ disengagement. Poor attendance, lack of motivation, disruptive behavior, irresponsibility 
regarding homework and grades are all symptoms of larger problems, bindings throughout this study 
point to such issues as: boring and repetitive instruction that fails to engage students intellectually; 
limited use of technology in classrooms to support engaged learning; expectations to attend tutorials 
outside the school day; repeated course failure, which narrows educational choices and opportunities; 
and poor access to advisement to help students set goals and see how current investments in learning 
yield future benefits. 

IMPLICATIONS FOR GRANT AWARDS AND MANAGEMENT 

Recommendations concerning grant management typically related to the timing of grant awards and 
funding. Many grantees appreciated efforts in later terms to streamline the evaluation process, bindings to 
follow relate to grant development, implementation and monitoring, and sustainability. 

Grant development. Grant applications should put greater emphasis on identifying problems, 
determining the root causes, and articulating how the project will alleviate those problems. NGSI grant 
development primarily involved campus and district administrators. Euture grant applications should be 
informed by the thinking of various stakeholders, with greater input from faculty, staff, and even parents 
and students. Grant programs for students at risk should also be aligned with curricular and learning 
expectations in regular classrooms. The establishment of separate or dual curricula for at-risk students in 
several NGSI schools conflicts with research demonstrating the harmful effects of tracking low- 
performing students (Oakes, 1985; Wheelock, 1992). Guidelines for grants should also lead districts and 
campuses to adopt research-based practices — thus, applicants should have access to research-based 
information on effective instruction and school improvement. Most importantly, grants aimed at 
improving learning and academic performance of at-risk students should include substantial investments 
in professional development, especially for classroom teachers. 
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Grant Implementation and monitoring. Grants should require or strongly encourage the addition of 
dedicated program leaders. Schools with dedicated program management at both the district and campus 
level appeared to have the greatest success implementing and continuing their grants. Major program 
changes made during the grant should also require TEA approval. Grant awardees should also have access 
to external technical support, assistance, and formative evaluation. Technical assistance hy external 
providers or agency staff broadens the pool of knowledge from which schools and districts can draw. 

Grant sustainability. Staff and administrator turnover undermined consistent grant implementation and 
had a negative impact on the continuation of NGSI programs; thus, districts should have a contingency 
plan to address changes in grant leadership. More widespread support for grant development and 
implementation will help to alleviate the void left when key project leaders leave a school or district. 
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Chapter 1 

INTRODUCTION AND METHODOLOGY 


INTRODUCTION 

This report presents findings from a comprehensive evaluation — conducted hy the Texas Center for Edu- 
cational Research for the Texas Education Agency — examining the effectiveness of three state-level pro- 
grams with the common goal of helping students at risk of failure to achieve academically. The Texas 
Study of Students at Risk (TxSSAR) comprises investigations of the Optional Extended Year Program, 
the Texas After School Initiative, and the Ninth Grade Success Initiative. Through a comprehensive 
evaluation (covering a four-year period between the 1999-2000 and 2002-03 school years), researchers 
explored ways in which state initiatives supported the academic success of at-risk students throughout 
their school careers. A brief summary of each program is presented below. 

Texas Study of Students at Risk 

Optional Extended Year Program. Eirst established by the 73rd Texas Eegislature in 1993, the Op- 
tional Extended Year Program (OEYP) is a state-funded program aimed at meeting the needs of elemen- 
tary and middle school students (grades K-8) who are at risk of not being promoted to the next grade 
level'. Eunds allow districts to provide an extended-year program for up to 30 instructional days for eligi- 
ble students, with the ultimate goal of reducing grade retention rates. Eligible students are those who are 
not likely to be promoted to the next grade level because they fail to meet district academic standards. 
During the four-year period evaluated, $191 million was provided to roughly 700 school districts. 

Texas After School Initiative. The Texas After School Initiative (TASI) for Middle Schools is a state 
initiative primarily designed to serve middle-school students (ages 10-14) at risk of academic failure 
and/or at risk of committing juvenile offenses. TASI funded after-school programs to accomplish three 
goals: (1) increase academic performance for participating students; (2) reduce referrals to the juvenile 
justice system; and (3) increase involvement of parents and/or mentors. Altogether, $36 million was allo- 
cated for TASI programs in 60 school districts. 

Ninth Grade Success Initiative. Under the Basic Skills Program for High School Students created by 
the 76th Texas Eegislature in 1999 and renewed in 2001, the state allocated $170 million to support 
school districts’ efforts to help ninth graders stay in school and succeed academically. The program, 
known as the Ninth Grade Success Initiative (NGSI), aimed to increase graduation rates in Texas public 
schools by reducing the number of students who either dropped out or were retained in ninth grade. 
Eunded programs were to emphasize basic skills in core curricular areas and provide targeted students 
with opportunities to build credits toward graduation. Targeted students included eighth graders who were 
advancing to ninth grade but were considered at risk academically, and ninth graders who had not 
earned — or were unlikely to earn — sufficient credit to advance to tenth grade and who failed to meet 
minimum skill levels. 

Eunded programs were expected to achieve four major objectives: (1) decrease the ninth-grade retention 
rate; (2) reduce the number of ninth-grade dropouts; (3) increase ninth-grade attendance rates; and 
(4) support successful performance on the state’s assessments — including the exit-level Texas Assess- 
ment of Academic Skills (TAAS) and its replacement, the Texas Assessment of Knowledge and Skills 
(TAKS). 


' In 2003, the 78th Legislature increased the range of the OEYP to serve grades K-12. Results for the 2003-04 
school year are beyond the scope of this evaluation. 


1 




study Approach 

This report provides detailed findings on the implementation and outcomes for all three programs, with 
information on services for at-risk students beginning in the primary grades (OEYP) and extending 
through middle school (TASI) and into high school (NGSI). A companion report — Texas Study of Stu- 
dents at Risk: Case Studies of Initiatives Supporting Ninth Graders’ Success — ^provides detailed findings 
from intensive case studies for 11 of 226 districts that received NGSI funding between the 1999-2000 and 
2002-03 school years. Conclusions and implications from NGSI case studies, which are included in this 
report, provide a greater understanding of issues facing large numbers of at-risk students, many of whom, 
despite potentially receiving services as early as kindergarten, still reach ninth grade unprepared to suc- 
ceed academically in high school. Many of these students end up repeating ninth-grade coursework or 
dropping out of school. 

Purpose of the Evaluation 

The overarching evaluation purpose was to assess the effective use of NGSI, TASI, and OEYP funds to 
improve student learning for targeted students. Evaluators aimed to determine what activities districts im- 
plemented to support students at risk, how funds supported student success, how activities can be sus- 
tained, and what models for best practice exist in projects. The study was guided by a theory of change 
model that identified pre-existing contextual factors that might influence project implementation and out- 
comes, program elements for projects, assumptions relevant to each program element, and measurable 
intermediate and long-term outcomes expected from initiatives. (See Theory of Change Model in Appen- 
dix A.) The model provided a framework to answer six broad research questions: 

• What are the characteristics of districts and students receiving grants, 

• How did grant resources supplement existing educational programs, 

• What is the effect of grant resources on targeted students, 

• What program elements are associated with student outcomes, 

• What best practices relative to grant projects exist, and 

• What are the implications for addressing the needs of students at risk? 

This comprehensive report presents available information relative to the research questions for each of the 
state initiatives. Researchers have collected information from grant recipients to gain a basic overview of 
project participants and activities. In addition to basic descriptive information on project activities, we 
conducted analyses to link available programmatic data to outcome measures in an attempt to understand 
how and why projects and activities succeed, for whom, and in what circumstances. The study employed 
both quantitative and qualitative methods to examine projects across four implementation years (1999-00, 
2000-01, 2001-02, and 2002-03). 

Organization of the Report 

Report findings are organized for each initiative around the primary research questions related to the ef- 
fective use of grant resources and the assessment of progress toward project goals. Specifically, results 
are presented for the OEYP, TASI, NGSI, and NGSI case studies: 

• Chapter 1 — Introduction and Methodology provides an overview of the evaluation purpose and ap- 
proach. In particular, general information is provided on data sources and data collection procedures. 
More specific methodological procedures are detailed in individual program chapters. 

• Chapter 2 — Optional Extended Year Program ( OEYP) provides an introduction to the OEYP, a brief 
review of research on extended-year programs, descriptive information relative to OEYP grantees and 
their students and the OEYP program, findings on student outcomes, and main conclusions and im- 
plications regarding the OEYP. 
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• Chapter 3 — Texas After School Initiative (TASI) includes an introduction to the TASI, a brief review 
of research on after-school programs, descriptive information relative to TASI grantees and their stu- 
dents and TASI program activities, findings on student outcomes, and main conclusions and implica- 
tions regarding the TASI. 

• Chapter 4 — Ninth Grade Success Initiative (NGSI) presents an introduction to the NGSI, a brief re- 
view of research on the ninth-grade problem, descriptive information relative to NGSI grantees and 
their students and NGSI program activities, findings on student outcomes, and main conclusions and 
implications regarding the NGSI. 

• Chapter 5 — Conclusions and Implications from Case Studies of Initiatives Supporting Ninth Graders’ 
Success offers findings organized around four areas: the NGSI program and existing best practices, 
evidence on the effect of grant resources on students, support for students at risk within the school 
context and educational environment, and recommendations for grant awards and management. 

METHODOLOGY 


Data Sources 

OEYP program reports from REIMS. Beginning in 1999-2000, districts receiving OEYP funds were 
required to submit student-level information regarding their extended-year programs to the Public Educa- 
tion Information Management System (PEIMS). Eor each student in the OEYP, the district submits data 
regarding: (a) student grade level; (b) the type of extended-year program (extended day, extended week, 
etc.); (c) the number of instructional days of the program; (d) the number of days absent and present; and 
(e) the promotion status of the student. Eor this report, OEYP records from PEIMS have been merged 
with TEA -provided demographic data and Texas Assessment of Academic Skills (TAAS) and Texas As- 
sessment of Knowledge and Skills (TAKS) scores. In addition, researchers reviewed a sample of district 
OEYP grant proposals. 

TASI and NGSI program and activity reports. TASI and NGSI grant recipients were required to 
submit program and activity reports to the TEA after each semester in which they served targeted stu- 
dents. Across four program years (1999-00 through 2002-03), districts were asked to submit two reports 
after each semester in which they served students. Eor TASI, the program report requested district-level 
information, such as general program information; activities supporting the academics-based, charac- 
ter/citizenship, and parent/mentor involvement components of TASI; staff participation and volunteer 
involvement; professional development; information dissemination; and district opinions regarding the 
most successful components. NGSI program reports requested district-level information, such as general 
program information, activities supporting credit recovery and basic skills, dissemination activities, staff 
participation and involvement, professional development, and district opinions regarding the most suc- 
cessful components. Procedures used to collect data on program reports were altered during spring 2003, 
and many data elements available in earlier terms were not collected during this term. 

In addition to the program reports, districts also submitted an activity report each semester with student- 
level data. Activity reports for TASI provided information for each program participant in seven areas: 
student demographics, student eligibility, retention, program attendance, additional activities, discipline 
referrals, and student performance. NGSI activity reports provided information in six areas for each par- 
ticipant: student demographic information, student eligibility, school attendance, retention and promotion, 
activities engaged in, and student performance. As with the program report, all information was self- 
reported. During the course of the program terms, activity report format changes also resulted in some 
data discontinuity. 

Student demographic and performance data. Researchers also gathered student-level data from the 
Texas Public Education Information Management System (PEIMS) and the Texas Academic Excellence 
Indicator System (AEIS). Student-level data supplied by participating school districts’ activity reports 
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were matched to PEIMS and AEIS data to create a set of master databases. Elements in the databases in- 
cluded student demographic information, such as ethnicity, gender, limited English proficiency (EEP) 
status, and grade level; Texas Assessment of Academic Skills (TAAS) and Texas Assessment of Knowl- 
edge and Skills (TAKS) scores, and attendance and promotion rates. 

Case studies of NGSI grants. Researchers also conducted case studies of a purposeful sample of NGSI 
grants to gain a greater understanding of issues facing students at risk. Although the original intent was to 
examine existing interrelationships among the three state-level funding streams for students at risk, it be- 
came evident early on that, in almost all cases, grants operated independently. Thus, case studies focused 
on NGSI projects and the broader high school contexts in which they operated. We conducted intensive 
studies of 11 of 226 districts that received NGSI funding between 1999-00 and 2002-03. In addition to 
NGSI funds, districts also benefited from OEYP or TASI funding, or both. Teams of two to three re- 
searchers visited each site between October 2003 and Eebmary 2004. Site visits included structured inter- 
views, focus groups, surveys, and classroom observations designed to collect information about the 
study’s primary research questions. Students participating in focus groups also completed a brief ques- 
tionnaire assessing their views on the school environment and plans for the future. A conceptual frame- 
work, formulated through a review of program objectives and recent research literature on recommended 
improvements in the nation’s high schools (e.g., American Youth Policy Eorum, 2000; High Schools that 
Work — Erome, 2001; NASSP, 1996/2003) provided the framework for the study. 
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Chapter 2 

OPTIONAL EXTENDED YEAR PROGRAM (OEYP) 


INTRODUCTION 

First established by the 73rd Texas Legislature in 1993, the Optional Extended Year Program (OEYP) is a 
state-funded program with the goal of meeting the needs of elementary and middle school students who 
are at-risk of not being promoted to the next grade level. Eunds allow districts to provide an 
extended-year program for up to 30 instructional days for eligible students, with the ultimate goal of 
reducing retention rates. Eligible students are those who are likely not to be promoted to the next grade 
level because they do not meet district academic standards. Originally established to serve first-grade 
students, legislators expanded the program in 1995 to include grades kindergarten through eight. School 
districts in which at least 35% of students in kindergarten through grade eight are from economically 
disadvantaged families are eligible for non-competitive grants to serve students in the OEYP. Districts’ 
level of funding is based on the amount necessary to offer extended-year services to not more than 10% of 
the at-risk student population in kindergarten through grade eight. 

State biennial funding for the program began with a $10 million appropriation by the 73rd legislature in 
1993 (Eigure 2.1). Subsequently, legislators increased two-year funding to $100 million in 1995, $113 
million in 1997, and to $121 million in 1999. The 77th legislature appropriated $116 million in 2001. In 
2003, the 78th legislature reduced biennial funding to $33 million and expanded the program to include 
grades K-12. 



Figure 2.1. State biennial funding for the OEYP, 1993-2003 
Organization of the Chapter 

Sections to follow include a brief literature review on extended-year programs and an overview of this 
study’s methodology. Additionally, OEYP program findings are presented for the following topics: 

(a) OEYP grantees and their students; (b) the OEYP program; (c) student outcomes for attendance, 
retention, and state-level assessments; (d) association of student- and district-level factors with academic 
achievement; (e) association of student- and district-level factors with retention; and (f) conclusions and 
implications. 
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RESEARCH ON EXTENDED-YEAR PROGRAMS 


Many states have policies that require extended-year programs or allow them to he provided as district 
options, with extended-year programs lengthening the traditional school year (Alper & Noie, 1987). 
Because of their pervasiveness, there is considerable variahility in eligibility criteria (e.g., individualized 
education program objectives, economically disadvantaged students, failing students), delivery options 
(days and hours of operation), student-teacher ratios, and faculty training requirements for extended-year 
services (Pinkerton, 1990). 

Several studies that have examined the effectiveness of extended-year programs have reported varied 
results. Heyns’s analysis of summer programs for at-risk students in Atlanta schools revealed gains in 
academic achievement, but at rates considerably slower than during the regular school year. The Extended 
School Year Program of the Detroit (Michigan) public schools found increased test scores for grade 4, 
and the majority of parents wanted the program to continue (Green, 1998). However, survey data 
indicated a decrease in teacher support for the program, and students said that they were not happy with 
the program. Yet about three-fourths of students and parents believed that the program increased students’ 
skills (Green, 1998). Results were mixed from an Austin, Texas study. While retention rates for 
extended-year students increased, student participation in the program also increased. Achievement 
results were positive with middle school students, but not with elementary school students (Washington, 
2000). 

Other studies have found benefits from extended-year programs. The reading and mathematics 
performance of New York City schools with and without extended time was compared. Students in 
extended-time schools improved at a greater rate on reading and mathematics assessments than did 
students in non-extended-time schools. (New York City Board of Education, 2000). In addition, 
extended-year programs may have a positive influence on kindergarteners. Erazier and Morrison (1998) 
found that extended-year attendees outperformed traditional-year students in mathematics, reading, and 
general knowledge and had higher perceived cognitive competence. A synthesis of extended-year 
programs listed outcomes realized by school districts (Sheane et al., 1994). These included decreased 
dropout rates, improved student achievement test scores, expanded extracurricular activities, reduced 
discipline problems, increased teacher/student employment opportunities, improved parent satisfaction, 
increased re-entry opportunities for at-risk students, and reduced taxpayer burdens. A paper by Worthen 
and Zsiray (1994) concluded that students in year-round and extended-year programs will maintain or 
improve their academic achievement, exhibit better attitudes toward school, improve their overall 
attendance, and drop out of school less often. 

Two studies asserted that successful extended-year programs require specific conditions. A meta-analysis 
of evaluations of summer programs found that student scores increased by an average of two-tenths of a 
standard deviation (Cooper, 2001). However, Cooper (2001) stated that summer programs are most 
effective if specific skills are taught that match what is tested. He also maintained that (a) middle-class 
students learn more than disadvantaged students, (b) mandatory summer school is more effective than 
voluntary summer school, and (c) factors like parent and community involvement as well as class size can 
influence the effectiveness of a summer program. A Southern Regional Education Board report (Denton, 
2002) recommended effective summer school programs for failing students. The report stated that 76% of 
over 75,000 failing students in North Carolina were promoted after attending summer school. None of 
over 165,000 failing students who did not attend summer school was promoted. Denton (2002) also 
pointed out that certain factors are needed to facilitate an effective summer program. These include 
(a) high-quality teachers, (b) adequate funding, (c) a focus on reading and mathematics, (d) a climate of 
innovation and creativity, and (e) a plan to evaluate program results. 

Glass (2002) has extensively studied allocation and duration of schooling. He found that small increases 
(10-15%) in time for schooling resulted in no significant achievement gains. Alternative ways of offering 
the 180 days of schooling (e.g., year-round) also showed no increased benefits for student learning over 
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the traditional 9-month calendar (Glass, 2002). Regarding summer school programs, Glass (2002) stated 
that “there is no reason not to expect - hut little research to support - that three months of summer school 
would result in the same rate of academic progress as any three months of the traditional academic 
calendar.” He cogently concluded, “The productivity of the schools is not a matter of the time allocated to 
them as much as it is a matter of how they use the time they already have.” 

These research efforts indicate that there is much variability in the structure and success of extended- year 
programs. However, given the correct set of facilitating conditions, extended-year programs may he able 
to boost student achievement, improve attitudes toward school, and result in more students staying in 
school. 


METHODOLOGY 

The purpose of this evaluation is to assess the effective use of OEYP funds to improve student learning 
for targeted students. Beginning in 1999-2000, districts receiving OEYP funds were required to submit 
student-level information regarding their extended-year programs to the Public Education Information 
Management System (PEIMS). Eor each student in the OEYP, the district submits data regarding: 

(a) student grade level; (b) the type of extended-year program (extended day, extended week, etc.); (c) the 
number of instructional days of the program; (d) the number of days absent and present; and (e) the 
promotion status of the student. Eor this report, OEYP records from PEIMS have been merged with TEA- 
provided demographic data and Texas Assessment of Academic Skills (TAAS) and Texas Assessment of 
Knowledge and Skills (TAKS) scores. While the analysis primarily used quantitative methods, selected 
document reviews were also utilized. 

Data sources included the following: 

• OEYP data such as students served and retention and promotion status (from end-of-year 
district-submitted program evaluation reports, PEIMS); 

• Individual student data including TAAS and TAKS scores provided by TEA; 

• AEIS data files for supplementation and comparisons; and 

• OEYP district grant proposals. 

OEYP data were summarized to show the characteristics of participating districts and students, the types 
of OEYP programs, and student participation across years. Student outcomes like retention rates and 
achievement on state-level assessments were tabulated and analyzed. Eor better understanding, program 
outcomes were broken down by relevant categorical variables, and student- and district-level factors 
associated with achievement and retention were explored. 

OEYP GRANTEES AND THEIR STUDENTS 

This section presents findings on the characteristics of districts receiving OEYP grants and the amount of 
awards paid to districts as well as the characteristics of students participating in the OEYP. 

Location of Districts Receiving Grants 

> OEYP grantees were distributed across aii of the ESC regions. 

Table 2.1 shows that the largest percentage of grantees was from Education Service Center (ESC) 7, the 
Kilgore region of East Texas. Other ESC regions with larger percentages of grantees included 6 
(Huntsville), 12 (Waco), 17 (Eubbock), and 20 (San Antonio). ESC regions with smaller percentages of 
grantees included 5 (Beaumont) and 19 (El Paso). Note that the distribution of grantees parallels the 
distribution of districts across ESC regions throughout Texas. 
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Table 2.1. Percentage of OEYP Grantees by ESC Region 


ESC 

ESC 

Percent of All 

Cohort 1 

Cohort 2 

Cohort 3 

Cohort 4 

Region 

Location 

Districts 

1999-00 

2000-01 

2001-02 

2002-03 

1 

Edinburg 

4.1 

5.2 

5.3 

5.4 

5.0 

2 

Corpus Christi 

3.9 

5.0 

4.8 

5.2 

5.3 

3 

Victoria 

3.3 

4.2 

4.1 

3.7 

3.9 

4 

Houston 

8.3 

5.3 

5.1 

5.2 

5.1 

5 

Beaumont 

2.9 

1.9 

2.2 

2.2 

2.6 

6 

Huntsville 

4.9 

6.0 

6.2 

6.4 

6.7 

7 

Kilgore 

8.4 

10.5 

10.1 

10.7 

10.8 

8 

Mt. Pleasant 

3.9 

4.2 

4.1 

4.2 

4.1 

9 

Wichita Palls 

3.3 

3.7 

4.1 

4.0 

3.5 

10 

Richardson 

9.2 

6.0 

5.7 

5.4 

5.6 

11 

Pt. Worth 

7.2 

3.9 

4.8 

4.5 

4.4 

12 

Waco 

7.0 

6.6 

7.3 

7.6 

7.0 

13 

Austin 

5.8 

4.9 

4.7 

4.6 

4.5 

14 

Abilene 

3.6 

4.6 

4.4 

4.5 

4.7 

15 

San Angelo 

3.6 

3.9 

3.7 

3.1 

3.9 

16 

Amarillo 

5.3 

6.2 

5.9 

5.4 

5.1 

17 

Lubbock 

5.1 

6.5 

6.3 

6.5 

6.4 

18 

Midland 

2.9 

3.5 

3.5 

3.6 

3.5 

19 

El Paso 

1.3 

1.6 

1.6 

1.8 

1.8 

20 

San Antonio 

6.0 

6.3 

6.0 

6.1 

6.0 


Source: TEA AEIS databases from 1999-00 through 2002-03. 

Note. The total number of district grantees for cohorts 1 through 4 were 695, 682, 672, and 684, 
respectively. 


Characteristics of Districts Receiving Grants 

> Nearly half of OEYP districts are very small (fewer than 1 ,000 students). However, compared to 
the state, fewer districts are very small and more are small (1 ,000 to 3,000 students) and mid- 
sized (3, 00^ to 10,000 students). 

In addition to the nearly half of OEYP districts being categorized as very small (Table 2.2), another 25% 
to 30% fall in the small category (enrollments from 1,000 to 3,000). Less than 10% of OEYP districts are 
large (enrollments from 10,001 to 25,000) or very large (enrollments more than 25,000). The size 
category breakdown of OEYP districts roughly parallels that of the state. However, compared to the state, 
fewer OEYP districts are very small, but more are small and midsize. 


Table 2.2. Percentage of OEYP Grantees by District Size 


District Size: 
Student Enrollment 

Percent of 
All 

Districts 

Cohort 1 
1999-00 

Cohort 2 
2000-01 

Cohort 3 
2001-02 

Cohort 4 
2002-03 

Very small: Pewer than 1,000 

59.6 

49.5 

47.9 

47.2 

47.5 

Small: 1,000 to 3,000 

20.4 

27.1 

27.6 

27.7 

26.2 

Midsize: 3,001 to 10,000 

12.9 

15.1 

16.0 

16.4 

17.3 

Large: 10,001 to 25,000 

3.8 

4.7 

4.8 

5.1 

4.8 

Very large: More than 25,000 

3.2 

3.6 

3.7 

3.7 

4.2 


Source: TEA AEIS databases from 1999-00 through 2002-03. 

Note. The total number of district grantees for cohorts 1 through 4 were 695, 682, 672, and 684, 
respectively. 







Awards Paid to Districts 

> The average award actually paid to districts was between $76,000 and $80,000. Paid awards 
ranged from $317 to over $5,000,000 (Table 2.3). 


Table 2.3. Summary Statistics of Awards Paid to Districts by Cohort 


Summary Statistic 

Cohort 1 
1999-00 

Cohort 2 
2000-01 

Cohort 3 
2001-02 

Cohort 4 
2002-03 

Mean 

$76,197 

$79,449 

$78,015 

$77,458 

Median 

$13,978 

$14,790 

$13,735 

$13,775 

Standard deviation 

$273,937 

$283,466 

$284,097 

$275,395 

Minimum 

$1,845 

$1,825 

$681 

$317 

Maximum 

$4,177,592 

$5,065,576 

$4,338,695 

$4,136,236 


Source: TEA OEYP financial data from 1999-00 through 2002-03. 

Note. The total number of awards paid for cohorts 1 through 4 was 696, 672, 661, and 626, respectively. 
In cohorts 2 through 4, there were 14, 11, and 57 awarded districts, respectively, that were paid $0. 


> Approximately 40% of districts received less than $10,000 for their OEYP program. 

The districts that received less than $10,000 (39% to 42%) correspond with nearly half of OEYP districts 
having enrollments of less than 1,000 and a minimum allocation per district of $5,500. About 80% of 
districts were paid less than $50,000, and approximately 10% were paid between $100,000 and $999,999. 
Only about 1% of districts received $1,000,000 or more (Table 2.4). 


Table 2.4. Percentage and Number of Award Payments by Size of Award and Cohort 



Cohort 1 

Cohort 2 

Cohort 3 

Cohort 4 

Size of Award 

1999-00 

2000-01 

2001-02 

2002-03 

Less than $10,000 

39.4% 

40.0% 

41.9% 

41.5% 

(274) 

(269) 

(277) 

(260) 

Greater than or equal to $10,000 but 

38.6% 

37.9% 

36.8% 

36.6% 

less than $50,000 

(269) 

(255) 

(243) 

(229) 

Greater than or equal to $50,000 but 

9.6% 

9.5% 

9.2% 

9.4% 

less than $100,000 

(67) 

(64) 

(61) 

(59) 

Greater than or equal to $100,000 but 

11.2% 

11.2% 

10.7% 

11.2% 

less than $1,000,000 

(78) 

(75) 

(71) 

(70) 

$1,000,000 or more 

1.1% 

1.3% 

1.4% 

1.3% 

(8) 

(9) 

(9) 

(8) 


Source: TEA OEYP financial data from 1999-00 through 2002-03. 
Note. Number of districts shown in parentheses. 


Characteristics of Students Participating in OEYP 

> About 190,000 students from nearly 700 school districts participated in the OEYP each year. 

In 1999-00, 191,335 students from 695 Texas school districts (58.7% of the 1,183 school districts in the 
state of Texas) participated in OEYP (Table 2.5). In 2000-01, 187,550 students from 682 districts (56.9% 
of the districts) participated, in 2001-02, 187,974 students from 672 districts (55.1% of the districts) 
participated, and in 2002-03, 217,471 students from 684 districts (55.9% of the districts) participated. 
Participants represented more than half of Texas’ approximately 1,200 school districts. 
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Table 2.5. OEYP Student Cohorts 


Cohort 

Participation 

Year 

Number of 
Districts 

Number of 
Students 

Percent of 
Districts 

1, 1999-00 

1999-2000 

695 

191,335 

58.7 

2, 2000-01 

2000-01 

682 

187,550 

56.9 

3, 2001-02 

2001-02 

672 

187,974 

55.1 

4, 2002-03 

2002-03 

684 

217,471 

55.9 


Source: TEA AEIS databases from 1999-00 through 2002-03. 


> OEYP participants are distributed across grades 1 through 8, with the largest proportion being 
third graders. 

In all four cohorts, students were distributed across grades 1 through 8, although grade 3 had the highest 
percentage of participants (Table 2.6). Between 1999-2000 and 2002-03, decreasing percentages of 
students from grades 1 and 2 participated in OEYP. 


Table 2.6. Grade Level Classifications for OEYP Student Cohorts 



Cohort 1 
1999-2000 

Cohort 2 
2000-01 

Cohort 3 
2001-02 

Cohort 4 
2002-03 

Grade Level 

N 

% 

N 

% 

N 

% 

N 

% 

1 

25,504 

13.3 

18,564 

9.9 

17,060 

9.1 

17,954 

8.3 

2 

22,865 

12.0 

20,786 

11.1 

17,965 

9.6 

19,297 

8.9 

3 

27,353 

14.3 

28,886 

15.4 

31,918 

17.0 

33,657 

15.5 

4 

24,230 

12.7 

25,208 

13.4 

27,590 

14.7 

32,582 

15.0 

5 

18,771 

9.8 

19,643 

10.5 

21,415 

11.4 

28,982 

13.3 

6 

21,915 

11.5 

22,429 

12.0 

22,179 

11.8 

25,725 

11.8 

7 

21,787 

11.4 

21,968 

11.7 

21,319 

11.3 

25,354 

11.7 

8 

20,497 

10.7 

22,063 

11.8 

21,597 

11.5 

25,236 

11.6 

Subtotal 

182,922 

95.6 

179,547 

95.7 

181,043 

96.3 

208,787 

96.0 

Grade missing 

8,413 

4.4 

8,003 

4.3 

6,931 

3.7 

8,684 

4.0 

Total 

191,335 

100.0 

187,550 

100.0 

187,974 

100.0 

217,471 

100.1 

No. of districts 

695 

58.7 

682 

56.9 

672 

55.1 

684 

55.9 


Source: Student cohort numbers and percentages were calculated from PEIMS individual student data. 

Note. The number of school districts in Texas for years 1999-00 through 2002-03 was 1,183, 1,199, 1,220, and 
1 ,224 respectively. 


> Compared to the state, OEYP served a greater proportion of Hispanic students, slightly more 
African-American students, and substantially less White students. OEYP students were also 
more likely to be economically disadvantaged and limited English proficient. 

Tables 2.7a through 2.7d display OEYP demographic information. Compared to the state population of 
students, the OEYP cohorts had higher percentages of Hispanic students (approximately 64% compared 
with approximately 40%), slightly more African-American students (approximately 18% compared with 
15%), and lower percentages of White students (approximately 17% compared with approximately 42%). 
OEYP students were also more likely to be economically disadvantaged (approximately 79% compared 
with approximately 56% across Texas) and limited English proficient (approximately 31% compared with 
approximately 15% across Texas). 


10 





Table 2.7a. Student Demographic Information for OEYP Cohort 1 


Student Group 

Cohort 1 (1999-00) 

State Average 
Percent 

Difference 

N Students 

Percent 

African American 

33,637 

17.9 

14.7 

3.2 

Hispanic 

119,952 

63.8 

39.4 

24.4 

White 

32,284 

17.2 

43.7 

-26.5 

Other 

2,161 

1.1 

2.2 

-1.1 

Economically disadvantaged 

143,379 

79.2 

55.3 

23.9 

Special education 

22,593 

12.5 

13.1 


Limited-English proficient 

57,682 

31.9 

15.0 

16.9 

ESL 

22,826 

12.1 

13.6 

-1.5 


Source: Student cohort numbers and percentages were calculated from PEIMS individual student data. State 
percentages were calculated from the 1999-00 AEIS data files, using only elementary and middle schools. 


Table 2.7b. Student Demographic Information For OEYP Cohort 2 


Student Group 

Cohort 2 (2000-01) 

State Average 
Percent 

Difference 

N Students 

Percent 

African American 

31,727 

17.1 

14.6 

2.5 

Hispanic 

120,334 

64.8 

40.4 

24.4 

White 

31,619 

17.0 

42.7 

-25.7 

Other 

2,061 

1.1 

2.3 

-1.2 

Economically disadvantaged 

140,958 

79.0 

55.4 

23.6 

Special education 

20,979 

11.8 

13.1 

-1.3 

Limited-English proficient 

55,042 

30.8 

15.1 

15.7 

ESL 

21,904 

11.8 

13.7 

-1.9 


Source: Student cohort numbers and percentages were calculated from PEIMS individual student data. State 
percentages were calculated from the 2000-01 AEIS data files, using only elementary and middle schools. 


Table 2.7c. Student Demographic Information For OEYP Cohort 3 


Student Group 

Cohort 3 (2001-02) 

State Average 
Percent 

Difference 

N Students 

Percent 

African American 

33,765 

18.0 

14.5 

3.5 

Hispanic 

118,816 

63.5 

41.4 

22.1 

White 

32,502 

17.4 

41.6 

-24.2 

Other 

2,125 

1.1 

2.5 

-1.4 

Economically disadvantaged 

143,568 

79.2 

56.3 

22.9 

Special education 

18,720 

10.3 

12.5 

-2.2 

Limited-English proficient 

54,211 

29.9 

15.7 

14.2 

ESL 

21,287 

11.4 

14.3 

-2.9 


Source: Student cohort numbers and percentages were calculated from PEIMS individual student data. State 
percentages were calculated from the 2001-02 AEIS data files, using only elementary and middle schools. 
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Table 2.7d. Student Demographic Information for OEYP Cohort 4 


Student Group 

Cohort 4 (2002-03) 

State Average 
Percent 

Difference 

N Students 

Percent 

African American 

40,639 

18.7 

14.5 

4.2 

Hispanic 

139,204 

64.2 

42.3 

21.9 

White 

34,370 

15.8 

40.6 

-24.8 

Other 

2,641 

1.2 

2.6 

-1.4 

Economically disadvantaged 

168,833 

80.2 

57.8 

22.4 

Special education 

19,299 

9.2 

12.4 

-3.2 

Limited-English proficient 

62,038 

29.5 

16.1 

13.4 

ESL 

25,436 

11.7 

14.8 

-3.1 


Source: Student cohort numbers and percentages were calculated from PEIMS individual student data. State 
percentages were calculated from the 2002-03 AEIS data files, using only elementary and middle schools. 


OEYP PROGRAM 

Information to follow describes the types of programs districts implemented with OEYP resources. Data 
come from reports submitted by all OEYP-funded districts through PEIMS and grant proposals for 
districts receiving funding during 2002-03 (cohort 4). 

OEYP Program Types 

> About two-thirds of OEYP students participated in an extended-year or intercession program 
only, whereas about one-fourth only participated in an extended-day program. 

Across cohorts, 85% to 90% of OEYP students were served by either an extended-day program or an 
extended-year or intercession program (Table 2.8). About 57% to 71% of OEYP students participated in 
just an extended-year or intercession program, whereas another 19% to 28% participated in only an 
extended-day program. Extended-week programs served 6% or less OEYP students, and participation 
rates for combination program types were less than 10%. 


Table 2.8. OEYP Program Type (Percent) by Year 


Program Type 

Cohort 1 
1999-00 

Cohort 2 
2000-01 

Cohort 3 
2001-02 

Cohort 4 
2002-03 

Extended day only 

19.1 

22.8 

27.6 

28.2 

Extended week only 

2.9 

4.1 

5.2 

6.0 

Extended year or intercession only 

70.5 

64.6 

58.7 

57.3 

Ext. day, week, & year or intercession 

0.8 

0.7 

0.8 

0.4 

Extended day & extended week 

2.0 

1.2 

1.4 

2.5 

Ext. day & ext. year or intercession 

3.6 

5.8 

5.2 

4.8 

Ext. week & ext. year or intercession 

1.1 

0.8 

1.1 

0.9 

Total 

100.0 

100.0 

100.0 

100.1 


Source: Program percentages were calculated from individual student data (PEIMS). 
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> Across four school years, the proportion of districts oniy impiementing extended-day and 
extended-week programs increased, whiie the prevaience of extended year or intersession 
oniy programs decreased. 

In 1999-00, about one-fifth (19%) of districts implemented extended-day programs only. By 2002-03, 
more than one-fourth of districts (28%) provided an extended-day program. In contrast, 7 out of 10 
districts (71%) implemented an extended-year or intersession program only in 1999-00, but the proportion 
of districts relying on such programs declined to somewhat more than half (57%) by the 2002-03 school 
year. 

Grade Levels and Content Areas Targeted 

Researchers analyzed grant proposals for cohort 4 (2002-03) in detail to determine the grade levels and 
content areas targeted by districts receiving grants. It was assumed that cohort 4 grant proposals were 
similar to grant proposals from previous years. Proposed instructional as well as parent involvement 
activities and professional development opportunities were tabulated from a sample of applications. 
Specifically, 19 grant applications (3%) were randomly selected from the population of 684 applications. 
The sample was stratified by district size with nine applications taken from very small districts (fewer 
than 1,000 students), five from small districts (1,000 to 3,000) districts, three from mid-size districts 
(3,001 to 10,000), one from large districts (10,001 to 25,000) districts, and one from very large districts 
(more than 25,000). 

> At least one third of the districts served students in grades 1 and 2, whereas neariy haif 
targeted each of grades 3 through 8. 

As Table 2.9 shows, a greater proportion of districts focus OEYP resources on students in upper grade 
levels. About one third of districts (33% to 35%) targeted students in grades 1 and 2. On the other hand, 
nearly half (43% to 46% focused on students in the upper grades (3 through 8). 

> in district grant appiications, mathematics and reading were the most frequentiy targeted 
content areas for both eiementary and middie schooi grades. 

Table 2.10 shows that OEYP programs most often targeted mathematics (86%), reading (78%), and 
English/language arts (56%). Districts also targeted science (39%), social studies (37%), and writing 
(16%) to a lesser extent. Additional analyses revealed that the distribution of content areas was similar for 
elementary grades (1 through 5) and middle grades (6 through 8). 


Tabie 2.9 Number and Percent of 
2002-03 OEYP Districts Targeting 
Each Grade Levei 


Grade 

Level 

Number^ 

Percentage 

1 

228 

33.3 

2 

245 

35.8 

3 

297 

43.4 

4 

311 

45.5 

5 

316 

46.2 

6 

314 

45.9 

7 

302 

44.2 

8 

299 

43.7 


Source: TEA Standard Application 
System OEYP applications for 2002-03. 
“Total number of cohort 4 districts 
was 684. 


Tabie 2.10. OEYP Districts Targeting Each 
Content Area, 2002-03 


Content 

Area 

Number^ 

Percentage 

Mathematics 

585 

85.5 

Reading 

534 

78.1 

English/Language Arts 

380 

55.6 

Science 

267 

39.0 

Social Studies 

255 

37.3 

Writing 

110 

16.1 


Source: TEA Standard Application System OEYP 
applications for 2002-03. 

“ Total number of cohort 4 districts was 684. 
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Proposed Instructional Activities 

> OEYP instructional activities focused most often on reading/language arts (e.g., reading 
strategies, computer-assisted instruction, TEKS) and mathematics (e.g., computer-assisted 
instruction, TEKS). 

Districts most frequently proposed OEYP-funded instructional activities for reading/language arts classes 
than for classes in other content areas (Table 2.11). A variety of reading strategies (e.g., choral reading, 
concept word maps, guided reading, word walls, thinking maps, word wizard, touch phonics, and phonic 
awareness) were proposed along with special programs like the New Century integrated instructional 
system, Reading Recovery, and Saxon Phonics. Computer-assisted instructional programs were 
frequently put forward for both reading and mathematics instruction. These included Accelerated Reader, 
the A-i-nyWhere Learning System, PLATO software, and SuccessMaker. Across all content areas, a 
common instructional theme was instruction in TEKS -deficient skill areas. 

> Districts mainly focused professional development opportunities on instructional strategies 
and strategies for teaching students at risk. 

Districts proposed a wide variety of professional development opportunities. About two-thirds proposed 
training in instructional strategies like integrating technology into instruction, questioning strategies, 
learning styles, cooperative learning, graphic organizers, and training on interdisciplinary instruction. 
Effective strategies for teaching students at risk was another popular area for training. Topics in this area 
included understanding cultural diversity and interventions to reduce failures. 

Note that while all of the districts planned professional development, approximately one-third (6 of 19 
sampled proposals or 32%) of the districts did not plan on using OEYP funds (Table 2.12). (Across all 
OEYP-funded districts, 55% proposed professional development funded by OEYP.) 

> Districts most frequently planned to involve parents through conferences, parent workshops, 
and various communication strategies. 

Numerous and varied parent involvement activities were planned, with parent conferences, parent 
workshops, general communication strategies, informational meetings, needs assessments (i.e., parent 
surveys), and home visits most frequently mentioned (Table 2.13). Other planned parent involvement 
activities included (a) letters and information packets explaining program opportunities, (b) telephone 
calls seeking input and providing progress reports, (c) developing compacts with parents, and (d) 
volunteer opportunities. 
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Table 2.11. Proposed Instructional Activities of OEYP Classes by Subject Area 


Activity 

Frequency 

Reading/Language Arts 


Reading strategies 

14 

Computer-assisted instruction 

9 

Instruction in TEKS skill areas 

7 

Special programs 

6 

Writer’s Workshop, writing process, creative writing, connecting reading/writing 

5 

Organizational approach (One-on-one, individualized, peer tutoring, small group) 

3 

Diagnostic/prescriptive instruction 

2 

Creation of writing samples using hooks, pictures, personal experience, etc. 

2 

Mathematics 


Computer assisted instruction 

8 

Instruction in TEKS skill areas 

7 

Manipulatives 

3 

Organizational approach (One-on-one, peer tutoring, group instruction) 

3 

Critical thinking/problem solving 

2 

Science 


Instruction in TEKS skill areas 

5 

Other (measurement, graphs, inferences, discovery, manipulatives, environment) 

5 

Social Studies 


Instruction in TEKS skill areas 

5 

Map skills, reference skills, use of timelines 

3 

Appreciation and study of cultures 

2 


Source: TEA Standard Application System OEYP applications for 2002-03. 


Table 2.12. OEYP Funded Professional Development Opportunities 


Professional Development Activity 

Frequency 

Instructional strategies 

12 

Effective strategies for students at-risk 

9 

Training on special programs 

3 

Using assessments 

2 

Other (parent involvement, Reading Academy, Training on TEKS) 

3 


Source: TEA Standard Application System OEYP applications for 2002-03. 


Table 2.13. OEYP Parent Involvement Activities 


Parent Involvement Activity 

Frequency 

Conferences with parents at various times throughout the OEYP program 

17 

Parent workshops 

14 

Communication strategies 

13 

General information meeting with parents 

7 

Survey of parents to determine needs 

6 

Home visits with parents as needed 

5 

Progress reports 

5 

Developing parent compacts 

3 

Celebration at the end of the extended-year program; student recognition 

2 

Volunteering 

2 


Source: TEA Standard Application System OEYP applications for 2002-03. 
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STUDENT OUTCOMES 


This section presents findings on the effect of grant resources on targeted students. To the extent possible, 
researchers also examine associations between OEYP program elements and student outcomes. We first 
present results for student participation and attendance and grade-level retention trends. Next, student 
performance on state-level assessments is compared by student cohort, participation in OEYP across 
school years, OEYP program type, and OEYP instructional days. Einally, the relationships between 
OEYP student-level and district-level variables and student academic achievement and grade-level 
retention are explored using complex statistical models. 

Participation and Attendance 

> On average, the number of OEYP instructional days available declined across years, with 20 
days in 1999-00, 17 in 2000-01, and 15 in both 2001-02 and 2002-03. Intercessions or extended- 
year programs had the largest number of instructional days. 

> Students’ OEYP attendance rates (81% to 86%) are considerably lower (Figure 2.2) than their 
attendance rates during regular school terms (about 96%). 

On average, 20 OEYP instructional days were available to Cohort 1 students, 17 to Cohort 2 students, and 
15 to Cohort 3 and cohort 4 students (Table 2.14). OEYP treatments involving intercessions or extended 
years resulted in the largest number of instructional days. Average rates of attendance for cohorts 1 
through 4 were 81%, 85%, 86%, and 86%, respectively. Of the two most typical program types 
(extended-day only and extended-year or intercession only), participation rates were higher for students in 
extended year or intercession (from 8 to 33 percentage points higher). 


Table 2.14. Mean OEYP Instructional Days, Days Absent, and Days Present by 
Program Type and Cohort 


Program Type 

Cohort 1 (1999-00) 

Cohort 2 (2000-01) 

Cohort 3 (2001-02) 

Cohort 4 (2002-03) 

Instr. 

Days 

Days 

Present 

Percent 

Days 

Present 

Instr. 

Days 

Days 

Present 

Percent 

Days 

Present 

Instr. 

Days 

Days 

Present 

Percent 

Days 

Present 

Instr. 

Days 

Days 

Present 

Percent 

Days 

Present 

Extended day only 

13.0 

7.0 

53.8 

8.5 

6.8 

80.0 

7.8 

5.9 

75.6 

7.4 

5.6 

79.7 

Extended week only 

8.2 

5.1 

62.2 

7.3 

5.1 

69.9 

7.0 

4.4 

62.9 

7.4 

4.9 

68.8 

Extended year or intercession only 

21.3 

18.4 

86.4 

19.8 

17.4 

87.9 

19.2 

17.2 

89.6 

19.7 

17.7 

89.9 

Ext. day, week, & year or intercession 

28.3 

20.9 

73.9 

19.0 

13.6 

71.6 

21.1 

18.2 

86.3 

18.7 

16.4 

87.0 

Extended day & extended week 

17.9 

10.3 

57.5 

13.8 

7.8 

57.4 

11.7 

8.5 

72.6 

9.0 

6.6 

78.5 

Ext. day & ext. year or intercession 

23.0 

18.0 

78.3 

20.9 

14.8 

70.8 

17.3 

14.3 

82.7 

21.3 

18.1 

85.2 

Ext. week & ext. year or intercession 

25.6 

19.4 

75.8 

25.5 

21.5 

84.3 

25.3 

21.4 

84.6 

26.0 

22.0 

84.2 

Total 

19.S 

15.7 

80.5 

16.7 

14.2 

85.0 

15.3 

13.2 

86.3 

15.4 

13.3 

86.4 


Source: Mean days were calculated from individual student data. 
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Figure 2.2. Attendance rates for extended-day, extended week, and 
extended year/intercession programs, 2000-2003 


> OEYP students’ attendance rates during the regular school year are comparable to state 
averages, remain relatively stable across years, and exceed OEYP attendance. 

Table 2.15 shows average school attendance rates of OEYP students along with comparable state figures. 
OEYP students’ attendance rates during the regular school year were similar to state rates for 1998-99 
through 2001-02. 


Table 2.15. Average Attendance Rates (Percent) for OEYP Student 
Cohorts by Year 


Year 

Cohort 1 
1999-00 

Cohort 2 
2000-01 

Cohort 3 
2001-02 

Cohort 4 
2002-03 

State 

1998-99 

95.7 

95.7 

95.7 

No data 

95.4 

1999-00 

95.7 

95.8 

95.9 

95.9 

95.6 

2000-01 

95.2 

95.5 

95.8 

95.9 

95.5 

2001-02 

94.7 

95.2 

95.6 

95.9 

95.6 

2002-03 

94.0 

94.4 

95.1 

95.7 

No data 


Source: Student cohort attendance rates were calculated from individual student data. 
State attendance rates were extracted from AEIS Profile reports. 

Note. A shaded cell denotes the OEYP participation year. 


Retention 

> Compared to state averages, retention rates of OEYP students are far higher for grades 1 
through 3 (about 17, 12, and 6 percentage points, respectively) but only slightly higher for 
students in grades 4 through 8 (about 2 percentage points). 

Table 2.16 reports retention rates for each OEYP cohort along with state averages. As one might expect, 
OEYP retention rates within each grade of each cohort were much higher than state retention rates for that 
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school year. Findings suggest that districts are more likely to use retention in the early grades (Figure 2.3) 
as a means to support student academic performance. 


Table 2.16. Percent Retained Students by Grade and Cohort 


Grade 

Level 

Cohort 1 (1999-00) 

Cohort 2 (2000-01) 

Cohort 3 (2001-02) 

Cohort 4 (2002-03) 

OEYP 

State 

OEYP 

State 

OEYP 

State 

OEYP 

State“ 

1 

18.3 

5.9 

21.1 

5.8 

25.5 

5.8 

25.1 

- 

2 

12.0 

3.0 

16.4 

3.1 

17.1 

3.5 

16.9 

- 

3 

7.1 

2.3 

9.1 

2.2 

00 

00 

2.5 

11.0 

- 

4 

4.3 

1.2 

4.9 

1.3 

4.5 

1.4 

5.0 

- 

5 

2.8 

0.7 

2.5 

0.8 

2.7 

0.8 

3.2 

- 

6 

4.2 

1.5 

3.9 

1.6 

3.6 

1.5 

4.6 

- 

7 

4.9 

2.9 

4.9 

2.8 

4.6 

2.5 

5.7 

- 

8 

3.3 

2.0 

3.2 

1.9 

2.9 

1.9 

3.7 

- 


Sources: Student percentages for cohorts 1, 2, and 3 were calculated from individual student data by comparing 
current grade with subsequent grade. Student percentages for cohort 4 were calculated from individual PEIMS 
OEYP promotion retention indicator codes. State non-special education percentages were extracted from AEIS 
Profile reports. 

“State-level data were unavailable for 2002-03. 



■Cohort 1 (1999-00) 


■Cohort 2 (2000-01) 


■Cohort 3 (2001-02) —X— Cohort 4 (2002-03) 


Figure 2.3. Student retention by selected grade and cohort 
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State-Level Assessments 


> Passing rates for all OEYP student cohorts are well below state averages. For cohorts 1 and 2, 
TAAS passing rate gains (for the year before to the year after full OEYP participation) exceed 
state gains, but TAAS-to-TAKS passing rate gains for cohort 3 are mostly less than state 
gains. Thus, the achievement gap was narrowed for cohorts 1 and 2, but not for cohort 3. 

TAAS passing rates of cohort 1 students (1999-00) were considerably below state passing rates in all 
subject areas and across all years (Table 2.17a). OEYP deficits ranged from 17 to 30 percentage points in 
reading, from 13 to 28 percentage points in mathematics, from 19 to 25 percentage points in writing, from 
15 to 27 percentage points in science, from 25 to 49 percentage points in social studies, and from 25 to 42 
percentage points in all tests taken. OEYP passing rate gains from 1999 (the year prior to OEYP 
participation) to 2002 were larger than state gains. However, initial OEYP passing rates were much lower 
than state averages. 

Passing rates for cohorts 2 (2000-01) and 3 (2001-02) were also considerably below state passing rates in 
all subject areas and across all years (Tables 2.17b and 2.17c). Cohort 2 passing rate gains from 2000 (the 
year prior to OEYP participation) to 2002 were larger than state gains in all content areas except writing. 
Cohort 3 passing rate gains from 2001 (the year prior to OEYP participation) to 2003 involved comparing 
TAAS (2001) passing rates with TAKS (2003) passing rates. Noting the different tests, cohort 3 passing 
rate gains were smaller (larger negative gains) than state gains in reading, mathematics, writing, and all 
tests taken, and larger in science (smaller negative gain) and social studies. 


Table 2.17a. TAAS Passing Rates of Cohort 1 Students by Content Area and Year 


Content Area 

1998-99 

OEYP 

1999-00 

2000-01 

2001-02 

1999-2002 

Change 

OEYP 

State 

OEYP 

State 

OEYP 

State 

OEYP 

State 

OEYP 

State 

Reading 

59.3 

86.5 

57.5 

87.4 

67.5 

88.9 

74.8 

91.3 

15.5 

4.8 

Mathematics 

60.5 

85.7 

59.8 

87.4 

73.3 

90.2 

79.9 

92.7 

19.4 

7.0 

Writing 

68.4 

88.2 

63.3 

88.2 

69.1 

87.9 

69.2 

88.7 

0.8 

0.5 

Science 

50.0 

87.1 

64.2 

88.2 

75.1 

91.8 

77.7 

93.0 

27.7 

5.9 

Social Studies 

21.1 

70.1 

34.6 

71.8 

44.3 

77.0 

58.5 

83.7 

37.4 

13.6 

All Tests Taken 

45.0 

78.3 

37.9 

79.9 

51.6 

82.1 

60.5 

85.3 

15.5 

7.0 


Source: TAAS passing rates were calculated from individual student data. State TAAS passing rates were extracted 
from AEIS Snapshot and Profile reports. 

Note. Shaded cells denote the OEYP participation year. 


Table 2.17b. TAAS Passing Rates of Cohort 2 Students by Content Area and Year 


Content Area 

1998-99 

1999-00 

OEYP 

2000-01 

2001-02 

2000-2002 

Change 

OEYP 

State 

OEYP 

State 

OEYP 

State 

OEYP 

State 

OEYP 

State 

Reading 

62.0 

86.5 

59.8 

87.4 

60.8 

88.9 

72.1 

91.3 

12.3 

3.9 

Mathematics 

63.1 

85.7 

61.5 

87.4 

66.4 

90.2 

78.3 

92.7 

16.8 

5.3 

Writing 

69.3 

88.2 

69.6 

88.2 

65.0 

87.9 

67.6 

88.7 

-2.0 

0.5 

Science 

77.8 

87.1 

51.3 

88.2 

72.7 

91.8 

77.2 

93.0 

25.9 

4.8 

Social Studies 

48.3 

70.1 

22.0 

71.8 

42.2 

77.0 

57.4 

83.7 

35.4 

11.9 

All Tests Taken 

48.4 

78.3 

46.1 

79.9 

43.2 

82.1 

57.7 

85.3 

11.6 

5.4 


Source: TAAS passing rates were calculated from individual student data. State TAAS passing rates were extracted 
from AEIS Snapshot and Profile reports. 

Note. Shaded cells denote the OEYP participation year. 
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Table 2.17c. TAAS and TAKS Passing Rates of Cohort 3 Students 
by Content Area and Year 


Content Area 

1999-00 

2000-01 

OEYP 

2001-02 

2002-03 (TAKS) 

2001-2003 

Change 

OEYP 

State 

OEYP 

State 

OEYP 

State 

OEYP 

State 

OEYP 

State 

Reading 

64.4 

87.4 

64.6 

88.9 

66.9 

91.3 

50.7 

85.6 

-13.9 

-3.3 

Mathematics 

63.6 

87.4 

69.0 

90.2 

73.0 

92.7 

42.2 

77.8 

-26.8 

-12.4 

Writing 

71.0 

88.2 

71.6 

87.9 

64.6 

88.7 

55.9 

86.3 

-15.7 

-1.6 

Science 

65.5 

88.2 

54.0 

91.8 

76.3 

93.0 

37.1 

70.7 

-16.9 

-21.1 

Social Studies 

44.4 

71.8 

25.3 

77.0 

55.7 

83.7 

74.1 

90.1 

48.8 

13.1 

All Tests Taken 

50.3 

79.9 

52.4 

82.1 

50.7 

85.3 

27.5 

67.4 

-24.9 

-14.7 


Source: TAAS and TAKS passing rates were calculated from individual student data. State TAAS and TAKS 
passing rates were extracted from AEIS Snapshot and Profile reports. 

Notes. Shaded cells denote the OEYP participation year. TAKS data are for 2002-03, and TAAS data are for prior 
years. 

Passing rates for cohort 4 (2002-03) were also well below state passing rates in almost all subject areas 
and across all years (Table 2.17d). Note that these data include TAKS passing rates for 2002-03. Cohort 4 
TAAS passing rates were generally lower than cohort 3 TAAS passing rates, yet cohort 4 2003 TAKS 
passing rates were higher than cohort 3 2003 TAKS passing rates. Passing rate gains are not reported for 
cohort 4 because data from the year following full OEYP participation, 2003-04, were not available. 


Table 2.1 7d. TAAS and TAKS Passing Rates of Cohort 4 Students 
by Content Area and Year 


Content Area 

1999-00 

2000-01 

2001-02 

OEYP 

2002-03 (TAKS) 

OEYP 

State 

OEYP 

State 

OEYP 

State 

OEYP 

State 

Reading 

51.5 

87.4 

56.4 

88.9 

61.4 

91.3 

57.0 

85.6 

Mathematics 

49.1 

87.4 

65.3 

90.2 

71.7 

92.7 

52.5 

77.8 

Writing 

62.7 

88.2 

68.0 

87.9 

63.6 

88.7 

64.4 

86.3 

Science 

100.0 

88.2 

50.0 

91.8 

55.9 

93.0 

43.0 

70.7 

Social Studies 

50.0 

71.8 

100.0 

77.0 

36.6 

83.7 

79.9 

90.1 

All Tests Taken 

31.8 

79.9 

44.1 

82.1 

50.8 

85.3 

36.4 

67.4 


Sources: TAAS and TAKS passing rates were calculated from individual student data. State TAAS 
and TAKS passing rates were extracted from AEIS Snapshot and Profile reports. 

Notes. Shaded cells denote the OEYP participation year. TAKS data are for 2002-03, and TAAS data 
are for prior years. 


Participation Across Years and Achievement 

> About 60% of OEYP students served in 1999-00 only participated that year, while about 40% 
participated at least one additional year. 

Of the 191,335 OEYP participants in cohort 1, 60% only participated in 1999-00, 15% participated in 
both 1999-00 and 2000-01, and 7% participated in both 1999-00 and 2001-02. Only 6% of the cohort 1 
students participated in 1999-00 and the two subsequent years, 2000-01 and 2001-02, and only 3% 
participated in 1999-00 and the three subsequent years, 2000-01, 2001-02, and 2002-03. 
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Table 2.18. Cohort 1 Student Participation in OEYP Across Years 


Years 

Number 

Percent 

1999-00 only 

114,615 

59.9 

1999-00 and 2000-01 

28,161 

14.7 

1999-00 and 2001-02 

12,734 

6.7 

1999-00, 2000-01, and 2001-02 

10,603 

5.5 

1999-00 and 2002-03 

9,702 

5.1 

1999-00, 2001-02, and 2002-03 

5,005 

2.6 

1999-00, 2000-01, and 2002-03 

4,927 

2.6 

1999-00, 2000-01, 2001-02, and 2002-03 

5,588 

2.9 

Total 

191,335 

100.0 


Source: Numbers and percentages were calculated from PEIMS individual 
student data. Includes 8,413 Cohort 1 students without valid grade 
designations. 


> As expected, lower performing students participated in OEYP for multiple years. With some 
exceptions, Cohort 1 students (1999-00) who participated in OEYP for more years had lower 
TAAS passing rates than their counterparts who attended fewer years. 

Table 2.19 shows that the highest performing cohort 1 students participated in OEYP only one year, while 
the lowest performing participated three or four years. 


Table 2.19. Cohort 1 TAAS Passing Rates and Retention Rate for 2002 
by Student Participation in OEYP 


Years 

TAAS 

Reading 

TAAS 

Math 

TAAS 

Writing 

TAAS 

Science 

TAAS 

Social 

Studies 

TAAS 

All 

Tests 

2002 

Retention 

Rate 

1999-00 only 

82.9 

85.3 

75.3 

83.0 

65.1 

68.9 

8.4 

1999-00 and 2000-01 

74.1 

80.5 

64.5 

74.6 

54.8 

59.9 

13.3 

1999-00 and 2001-02 

64.2 

70.3 

54.9 

72.3 

50.2 

44.1 

5.7 

1999-00, 2000-01, and 2001-02 

60.7 

70.1 

51.4 

65.7 

44.2 

42.6 

5.5 

1999-00 and 2002-03 

70.8 

78.5 

70.3 

60.0 

35.6 

62.0 

3.1 

1999-00, 2001-02, and 2002-03 

55.1 

64.3 

62.4 

35.7 

25.0 

41.2 

4.0 

1999-00, 2000-01, and 2002-03 

65.2 

76.0 

62.9 

65.3 

39.6 

56.0 

4.6 

1999-00 through 2002-03 

52.0 

64.6 

60.3 

43.8 

52.9 

39.8 

4.3 

All Years 

74.8 

79.9 

69.2 

77.7 

58.5 

60.5 

8.2 
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Program Type and Achievement 

> Students who only participated in extended-day or extended-week programs had higher TAAS 
and TAKS scores and lower retention rates than students enrolled in extended-year or 
intercession programs. 

Tables 2.20a through 2.20c report 2002 TAAS passing rates and 2002 retention rates for cohorts 1, 2, and 
3. Across all cohorts, students who participated in extended-day or extended-week programs had the 
highest TAAS scores and the lowest retention rates. Students who participated in 
extended-year/intercession programs had the lowest TAAS scores and the highest retention rates. 

Table 2.20d reports 2003 TAKS passing rates and 2003 retention rates for cohort 4. As with the other 
cohorts, extended-day or extended-week participants had the highest TAKS scores and the lowest 
retention rates. Extended-year/intercession participants had the lowest TAKS scores and the highest 
retention rates. Even so, we cannot infer from these data that participation in after-school programs causes 
improved student achievement or reduced retention. It is plausible that students who only participate in 
after-school programs may differ from those who are enrolled in extended-year programs or intercessions. 
It is possible that lower achievers may attend summer school or receive remedial assistance during 
intercessions. 


Table 2.20a. Cohort 1 TAAS Passing Rates and Retention Rate for 2002 
by OEYP Program Type 


Program Type (1999-00) 

TAAS 

Reading 

TAAS 

Math 

TAAS 

Writing 

TAAS 

Science 

TAAS 

Social 

Studies 


2002 

Retention 

Rate 

Extended day only 

80.2 

85.5 

74.8 

81.9 

61.4 

68.6 

5.5 

Extended week only 

76.1 

83.6 

72.8 

79.5 

64.0 

66.5 

3.5 

Extended year or intercession only 

72.8 

77.5 

67.5 

76.7 

57.6 

57.3 

9.1 

All Programs 

74.8 

79.9 

69.2 

77.7 

58.5 

60.5 

8.1 


Table 2.20b. Cohort 2 TAAS Passing Rates and Retention Rate for 2002 
by OEYP Program Type 


Program Type (2000-01) 

TAAS 

Reading 

TAAS 

Math 

TAAS 

Writing 

TAAS 

Science 

TAAS 

Social 

Studies 

TAAS 

All 

Tests 

2002 

Retention 

Rate 

Extended day only 

78.0 

85.1 

77.0 

82.3 

64.5 

68.1 

4.7 

Extended week only 

77.7 

85.5 

78.2 

79.7 

66.8 

68.8 

4.2 

Extended year or intercession only 

69.1 

74.3 

62.8 

76.1 

55.7 

52.0 

11.1 

All Programs 

72.1 

78.3 

67.6 

77.2 

57.4 

57.7 

9.1 


Table 2.20c. Cohort 3 TAAS Passing Rates and Retention Rate for 2002 
by OEYP Program Type 


Program Type (2001-02) 

TAAS 

Reading 

TAAS 

Math 

TAAS 

Writing 

TAAS 

Science 

TAAS 

Social 

Studies 

TAAS 

All 

Tests 

2002 

Retention 

Rate 

Extended day only 

80.9 

85.7 

78.4 

84.4 

65.7 

70.4 

3.9 

Extended week only 

79.8 

85.2 

78.7 

82.5 

72.3 

67.4 

3.4 

Extended year or intercession only 

58.0 

64.6 

55.1 

74.2 

52.4 

38.0 

9.5 

All Programs 

66.9 

73.0 

64.6 

76.3 

55.7 

50.7 

7.6 
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Table 2.20d. Cohort 4 TAKS Passing Rates and Retention Rate for 2003 
by OEYP Program Type 


Program Type (2002-03) 

TAKS 

Reading/ 

ELA 

TAKS 

Math 

TAKS 

Writing 

TAKS 

Science 

TAKS 

Social 

Studies 

TAKS 
All Tests 

2003 

Retention 

Rate 

Extended day only 

68.7 

72.8 

76.0 

53.8 

86.1 

56.9 

4.1 

Extended week only 

66.9 

69.8 

75.5 

48.8 

84.7 

54.2 

3.8 

Extended year or intercession only 

51.4 

38.9 

56.3 

33.6 

78.7 

22.8 

10.9 

All Programs 

57.0 

52.5 

64.4 

43.0 

79.9 

36.4 

8.5 


Instructional Days and Achievement 

> Students with the least academic need appear to participate in OEYP for a smaller number of 
instructional days. In the year following full OEYP participation, higher TAAS and TAKS 
passing rates are associated with lower percentages of OEYP prescribed instructional days 
attended. 

Table 2.21a presents cohort 1 TAAS passing rates and retention rates for 2001 stratified by percentage of 
OEYP days attended. Table 2.21b presents similar 2002 data for cohort 2. Table 2.21c shows cohort 3 
data which includes 2003 TAKS passing rates and 2003 retention rates. There is a tendency for higher 
TAAS and TAKS passing rates in the lower attendance categories. This may indicate that students who 
need less supplementary instruction attend less often. Interestingly, cohort 3 data show higher retention 
rates are also associated with lower percentages of OEYP prescribed instructional days attended. 


Table 21a. Cohort 1 TAAS Passing and Retention Rates by Percent OEYP 
Instructional Days Present 


Cohort 1 (1999-00) 
Outcome for 2001 

0 to 49% 
of Days 

50% to 74% of 
Days 

75% to 89% of 
Days 

90% to 100% 
of Days 

TAAS Reading 

71.5 

70.3 

67.3 

66.1 

TAAS Math 

76.7 

76.5 

73.3 

72.0 

TAAS Writing 

72.5 

71.1 

69.2 

67.8 

TAAS Science 

76.6 

76.8 

73.9 

74.8 

TAAS Social Studies 

46.3 

45.6 

40.7 

44.5 

TAAS All Tests Taken 

56.4 

55.0 

51.9 

49.8 

Retention Rate 

8.6 

9.0 

9.8 

9.0 


Table 2.21b. Cohort 2 TAAS Passing Rates and Retention Rates by Percent OEYP 
Instructional Days Present 


Cohort 2 (2000-01) 
Outcome for 2002 

0 to 49% 
of Days 

50% to 74% of 
Days 

75% to 89% of 
Days 

90% to 100% 
of Days 

TAAS Reading 

75.8 

74.2 

71.3 

71.4 

TAAS Math 

82.6 

80.2 

78.0 

77.4 

TAAS Writing 

71.6 

72.2 

66.7 

66.6 

TAAS Science 

77.7 

77.7 

76.4 

77.2 

TAAS Social Studies 

56.7 

59.1 

56.0 

57.6 

TAAS All Tests Taken 

63.1 

61.1 

57.8 

56.4 

Retention Rate 

7.7 

00 

00 

9.5 

9.2 
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Table 2.21c. Cohort 3 TAKS Passing and Retention Rates by Percent OEYP 
Instructional Days Present 


Cohort 3 (2001-02) 
Outcome for 2003 

0 to 49% 
of Days 

50% to 74% of 
Days 

75% to 89% of 
Days 

90% to 100% 
of Days 

TAKS Reading/ELA 

59.2 

54.3 

50.7 

48.8 

TAKS Math 

47.8 

46.6 

42.9 

40.5 

TAKS Writing 

63.5 

60.9 

56.8 

53.7 

TAKS Science 

43.3 

39.7 

40.5 

34.5 

TAKS Social Studies 

79.2 

76.5 

69.9 

74.0 

TAKS All Tests Taken 

34.4 

30.5 

28.3 

25.6 

Retention Rate 

10.6 

8.4 

8.2 

6.8 


Association of Student-and District-Levei Factors 
with Academic Achievement 

The relationships between OEYP student and district characteristics and TAAS reading and mathematics 
TLI scores were explored using hierarchical linear modeling (HEM). HEM was the preferred analytical 
method because, in most cases, students within school districts are similar to each other because of 
selection processes and common backgrounds. Consequently, measures within school districts are not 
independent. Rather, the correlation between measures of students from the same district will tend to be 
higher than the correlation between measures of students from different districts. Not only does 
hierarchical linear modeling make no assumption about independence, it estimates the degree of 
dependence of measures and uses this estimate in the calculation of the precision with which treatment 
effects are estimated. 

Separate HEM analyses were conducted using OEYP participants in 1999-00 (cohort 1) and 2000-01 
(cohort 2). Each group included over 40,000 OEYP students from over 400 school districts. These 
students participated in extended-day or extended-year/intercession programs in their respective OEYP 
year, constituted at least 10 students in the school district, and had TAAS scores from the year prior to 
OEYP participation (1999 for cohort 1 and 2000 for cohort 2) and the year after OEYP participation 
(2001 for cohort 1 and 2002 for cohort 2). These students were used to investigate the effect of OEYP 
student and district characteristics on TAAS reading and mathematics TEI scores. 

The specific student- and district-level variables along with their descriptive statistics are reported in 
Table 2.22 for TAAS reading and mathematics. The student-level variables included gender (1 if male, 0 
if female), economic status (1 if disadvantaged, 0 if not), prior retention (1 if retained prior to OEYP, 0 if 
not), school attendance (average rate for OEYP participation year), TAAS reading and mathematics TEI 
pretest scores (1999 for cohort 1 and 2000 for cohort 2), days taught in OEYP, OEYP program type (1 if 
extended day, 0 if extended year/intercession), minority status (1 minority, 0 if white), and grade group in 
the OEYP participation year (1 if grades 6 through 8, 0 if grades 3 through 5). 

District-level variables included the district OEYP mean TAAS pretest score (achievement contextual 
effect), the district proportion of disadvantaged OEYP students (economically disadvantaged contextual 
effect), and the district OEYP per pupil expenditure for that year. 
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Table 2.22. Descriptive Statistics for TAAS Reading and Mathematics Data 


TAAS Mathematics 

Maximum 

Cohort 1 (1999-00) Student-Level Descriptive Statistics 

o 

p 

o 

p 

o 

p 

o 

p 

o 

p 

93.00 

93.00 

30.00 

o 

o 

d 

o 

o 

p 

Cohort 1 (1999-00) District-Level Descriptive Statistics 

o 

p 

85.77 

687.05 

Cohort 2 (2000-01) Student-Level Descriptive Statistics 

o 

p 

o 

p 

o 

p 

o 

p 

o 

p 

93.00 

93.00 

30.00 

o 

o 

d 

o 

o 

p 

Cohort 2 (2000-01) District-Level Descriptive Statistics 

o 

p 

87.24 

693.39 
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Student-Level Variables 

> After controlling for the effect of student-level characteristics (academic and social 
background), extended-day participants had higher TAAS reading and mathematics scores 
than extended-year/intercession participants. 

> There was no positive relationship between days students spend in OEYP (up to 30 
instructional days) and TAAS scores. 

> A student’s school attendance rate had a stronger influence on TAAS mathematics scores 
than on TAAS reading scores. 

Table 2.23 shows that student-level predictors associated with higher TAAS reading scores include being 
female, economically advantaged, non-minority, not retained prior to OEYP, and attending grades 6 
through 8. Higher TAAS reading pretest scores were also strongly associated with higher posttest reading 
scores. For example, for cohort 1 OEYP students, a unit increase in 1999 TAAS reading scores was 
associated with a 0.49 unit increase in 2001 TAAS reading scores. Other student characteristics being 
equal, the average TAAS posttest reading scores of extended-day participants were 1.43 TEI units higher 
in cohort 1 and 1.28 TEI units higher in cohort 2 than the average posttest reading scores of extended- 
year/intercession participants. In addition, there was no significant positive relationship between OEYP 
instructional days (up to 30 days per student) and TAAS reading TEI scores. This is in line with findings 
that small increases (10-15%) in schooling time show no appreciable gains in achievement (Glass, 2002). 

Similar student-level predictors were associated with higher TAAS mathematics scores (Table 2.23). 
These included being female, economically advantaged, non-minority, not retained prior to OEYP, and 
having a higher TAAS mathematics pretest score. Unlike TAAS reading, higher mathematics scores were 
associated with attending grades 3 through 5, and lower scores were associated with attending grades 6 
through 8. School attendance was a stronger predictor of mathematics scores than reading scores, 
although the impact was small. For example, in cohort 1 , a 1 % increase in the attendance rate resulted in a 
0.09 TEI unit increase in mathematics posttest scores, net of other level 1 predictors. As with reading, 
extended-day participants were scoring about 1.2 TEI units (1.22 in cohort 1 and 1.15 in cohort 2) higher 
than extended-year/intercession participants with academic and social backgrounds similar to their own. 
Also, as with the reading analyses, there was not a positive relationship between OEYP instructional days 
and subsequent TAAS mathematics scores. Note that the coefficients for OEYP instructional days were 
uniformly small and negative across the content areas and cohorts. 

District-Level Variables 

> After controlling for the effect of student characteristics (academic and social background) 
and district social context and OEYP per pupil expenditures, OEYP students having higher 
achieving classmates performed better in TAAS reading and mathematics. 

> There was no significant relationship between OEYP dollars per pupil and TAAS reading and 
mathematics scores. 

Contextual effects occur when the aggregate of a student-level characteristic is related to an outcome 
variable, even after controlling for the effect of the student-level characteristic. An example would be 
average social class of a school being related to achievement after controlling for individual students’ 
social class (Raudenbush and Bryk, 2002). HEM facilitates this type of multi-level analysis by specifying 
separate student and district equations. Following this model. Table 2.23 shows that there was a 
significant district-level contextual effect for both reading and mathematics. The achievement level of 
OEYP students has a positive impact on subsequent OEYP district TAAS reading and mathematics 
scores. For example, in cohort 1, a 1.2 standard deviation increase (6.2 TEI units) in average reading 
pretest scores was associated with a 1-unit increase in a district’s reading posttest scores, after controlling 
for student-level variables. Eikewise, in cohort 2, a 1.7 standard deviation increase (8.3 TEI units) in 
mathematics pretest scores was associated with a 1-unit increase in a district’s mathematics posttest 
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scores. Finally, average OEYP dollars spent per student did not have a significant effect on district 
reading or mathematics scores. 


Table 2.23. HLM Analyses of TAAS Mathematics and Reading TLI Scores of 
Cohort 1 (1999-00) and Cohort 2 (2000-01) OEYP Students 


Conditional Model 

Reading 

Mathematics 

Cohort 1 

Cohort 2 

Cohort 1 

Cohort 2 

Student-Level Variables 

District-Level Variables 

Gamma 

Coefficient/ 

it) 

Gamma 

Coefficient/ 

(0 

Gamma 

Coefficient/ 

if) 

Gamma 

Coefficient/ 

it) 

Intercept 


75.87 

(586.50**) 

77.38 

(594.88**) 

76.05 

(665.25**) 

77.31 

(682.46**) 


Economically 
disadvantaged context 

1.02 

(1.11) 

-1.03 

(-1.08) 

0.20 

(0.25) 

-1.14 

(-1.43) 


Achievement context 

0.16 

(4.91**) 

0.12 

(3.94**) 

0.15 

(5.53**) 

0.12 

(4.11**) 


OEYP per pupil 
expenditure 

-0.0013 

(-1.25) 

-0.0008 

(-0.86) 

-0.0016 

(-1.76) 

-0.0006 

(-0.69) 

Gender 


-0.61 

(-3.56*) 

-0.25 

(-2.18) 

-0.44 

(-3.62*) 

-0.09 

(-1.10) 

Economically 

disadvantaged 


-1.19 

(-8.13**) 

-1.35 

(-10.19**) 

-0.37 

(-3.35*) 

-0.28 

(-2.69*) 

Prior retention 


-2.31 

(-12.89**) 

-2.40 

(-13.84**) 

-1.89 

(-13.69**) 

-2.02 

(-15.34**) 

School attendance 


0.04 

(2.60) 

0.01 

(0.82) 

0.09 

(8.44**) 

0.07 

(4.87**) 

TAAS pretest 


0.49 

(57.98**) 

0.48 

(58.90**) 

0.41 

(52.62**) 

0.39 

(49.99**) 

Days taught in OEYP 


-0.06 

(-1.96) 

-0.02 

(-1.64) 

-0.04 

(-1.73) 

-0.02 

(-1.04) 

OEYP type 


1.43 

(3.11*) 

1.28 

(3.73**) 

1.22 

(3.14*) 

1.15 

(4.36**) 

Minority 


-0.74 

(-4.14**) 

-1.43 

(-8.83**) 

-0.06 

(-0.45) 

-0.64 

(-5.71**) 

Grade group in OEYP 
year 


1.53 

(6.72**) 

1.09 

(4.56**) 

-3.93 

(-18.63**) 

-4.53 

(-21.08**) 


Notes. All continuous variables without a true 0 were grand-mean centered. In the student-level model, the 
intercept was specified as random and the independent variables were specified as fixed. The intraclass 
correlation coefficients were 0.09 (cohort 1 reading), 0.11 (cohort 1 math), 0.08 (cohort 2 reading), and 0.12 
(cohort 2 math). Compared to a null model, the student-level model resulted in a within-district variance 
reduction of 30.1% in cohort 1 reading, 29.3% in cohort 1 math, 27.7% in cohort 2 reading, and 28.0% in cohort 
2 math. The district-level model, conditional on the student-level model, resulted in a between-district variance 
reduction of 14.2% in cohort 1 reading, 14.4% in cohort 1 math, 13.9% in cohort 2 reading, and 1 1.5% in cohort 
2 math. 

*p< .01; **p< .001. 
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Association of Student-and District-Levei Factors 
with Retention 

Relationships between OEYP student and district characteristics and retention status (a binary outcome, 
retained or not retained) were analyzed using a hierarchical generalized linear model (HGLM) with a 
Bernoulli sampling model, a log odds or logit link function, and level- 1 (student) and level-2 (district) 
structural models identical to those in HLM. HGLM presents results for both unit-specific and 
population-average models. The unit-specific model holds constant school district attended, while the 
population- average model does not, but averages over all districts. Because the average log-odds of 
retention was found to vary significantly across districts (variance in district average log-odds of retention 
= 0.81 for cohort 1 and 0.83 for cohort 2, with significant chi-square values in both cases), this variation 
should be controlled or held constant. Consequently, only unit-specific results will be presented and 
discussed below. (Note, however, that results are similar for both models.) 

Included in separate HGLM analyses were students who participated in OEYP in 1999-00 (cohort 1) and 
2000-01 (cohort 2). Each group included about 70,000 OEYP students from about 500 school districts. 
The OEYP students participated in extended-day or extended-year/intercession programs in their 
participation year and constituted at least 10 students per school district. These data were used to 
investigate the effect of OEYP student and district characteristics on retention status (2000 retention 
status for cohort 1 and 2001 retention status for cohort 2). 

As previously defined, student characteristics included gender, economic status, prior retention, OEYP 
program type, days taught in OEYP, minority status, and grade group in the OEYP participation year. 
School attendance in the year of OEYP participation and the average of the TAAS reading and 
mathematics TLI scores in the year of participation were also used as student-level predictors (see Table 
2.24). District-level variables included the district OEYP mean of the TAAS combined reading and 
mathematics score (achievement contextual effect), the district proportion of disadvantaged OEYP 
students, and the district OEYP per pupil expenditure for the appropriate cohort year. 

Student-Level Variables 

> After controlling for the effect of student-level characteristics (academic and social 
background), an increase in the school attendance rate decreased the chances of retention. 

> For otherwise similar students, more instructional days in OEYP decreased the chances of 
retention for extended-year/intercession participants. 

> For otherwise similar students, fewer instructional days in OEYP decreased chances of 
retention for extended-day participants. 

As one might expect, being male and economically disadvantaged were associated with higher odds^ of 
retention (Table 2.25). Having higher TAAS scores, being in the middle-school grades, and having been 
retained prior to OEYP participation were associated with lower odds of retention. More importantly, 
school attendance was related to lower retention. A 1 % increase in the attendance rate (above the average 
attendance rate) corresponded to a 0.07 reduction in the log-odds of retention in cohort 1 and to a 0.05 
reduction in cohort 2. These log-odds of retention corresponded to odds of 0.93 and 0.95, respectively. If, 
for example, there are two otherwise similar cohort 1 students but one has a 1 % higher attendance rate, 
the odds of retention for that student are 0.93 times the odds for the student without the additional 
attendance. In other words, the student with the 1 % higher attendance rate will have a slightly smaller 
chance of being retained. 


* Odds are the probability that an event will occur divided by the probability that the event will not occur. If the 
weather forecast says that there is a 20% chance of rain, then there is an 80% chance that it will not rain, and the 
odds of rain are 0.20/0.80 or 0.25. If the forecast says that there is a 50% chance of rain, then there is also a 50% 
chance that it will not rain, and the odds of rain are 0.50/0.50 or 1 .00. If the forecast says that there is an 80% chance 
of rain, then there is a 20% chance that it will not rain, and the odds of rain are 0.80/0.20 or 4.00. 
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The effect of the numher of instructional days in the OEYP program varied across program types. That is, 
there was a significant interaction (coefficient = 0.08, t - 7.82 in cohort 1, and coefficient = 0.07, t - 5.95 
in cohort 2). The relationship between OEYP days taught and the log-odds of retention was stronger for 
extended-year/intercession participants. Other factors being equal, for extended-year/intercession 
participants each additional day of instruction in OEYP resulted in a -0.06 reduction in the log-odds of 
retention in cohort 1 and a -0.07 reduction in cohort 2. However, for extended-day participants, each 
additional day of instruction in OEYP resulted in a 0.02 increase in the log-odds of retention in cohort 1 
and neither an increase or a decrease in cohort 2 [days taught coefficient -i- 1 times interaction coefficient 
or-0.07 + (l)(0.07) = 0]. 

The interaction between OEYP days taught and program type means that the program type effect depends 
on the days taught. In cohort 1, when OEYP days taught were 17 or fewer, the log-odds of retention 
favored extended-day participants, and when OEYP days taught were 18 or more (with a maximum of 
30), the log-odds favored extended-year/intercession participants. Eor example, if OEYP days taught were 
10, the extended-day participants would have 0.54 times (exp(-0.62}) the odds of retention (a smaller 
chance) of otherwise similar students in the extended-year/intercession program. Yet if OEYP days taught 
were 25, the extended-day participants would have 1.79 times (exp(0.58}) the odds of retention (a greater 
chance) of otherwise similar students in the extended-year/intercession program. 

In cohort 2, similar results were found. When OEYP days taught were 22 or fewer, extended-day 
participants were favored (smaller retention chances). When OEYP days taught exceeded 22, 
extended-year/intercession program students were favored (smaller retention chances). 

OEYP extended-day programs may include homework assistance, tutorials, and remedial study. Students 
having less of a need for such help may attend fewer days. This may result in fewer instructional days 
being associated with decreased chances of retention. On the other hand, OEYP extended-year programs 
may have a fixed number of days of scheduled instruction for students who are in danger of being 
retained. Successful completion of the program may preclude retention. Those OEYP students who attend 
more of the scheduled days would certainly be expected to have lower chances of retention. 

District-Level Variables 

> After controlling for the effect of student characteristics (academic and social background) 
and district social and academic contexts, higher OEYP per-pupil expenditures were 
associated with increased student retention. 

The contextual variables of social class and overall achievement level did not influence district retention 
(Table 2.25). However, retention was negatively related to OEYP dollars per pupil after controlling for 
student demographic and performance variables as well as for district social class and achievement 
contexts. (Note that when the square root of the district size was added as a control variable, the 
coefficients for OEYP expenditure per pupil were essentially identical to the values shown in Table 2.25.) 
Using cohort 2 as an example, two otherwise similar districts differing by $100 in OEYP per pupil 
expenditure could be expected to be 0.22 units apart in log-odds of retention, or relative odds of 1.25. 
Consider the cohort 2 elementary white female who was not economically disadvantaged, had no prior 
retention, was in the extended year/intercession program, had average school and OEYP attendance, 
average TAAS scores, and attended a typical school district. She had a predicted log-odds of retention of 
-3.47, or a probability of 0.030. Adding $100 to the district per pupil expenditure would result in 
predicted log-odds of -3.25, or a predicted probability of 0.037. Adding an additional $100 would 
increase the predicted probability of retention to 0.046. Thus, higher OEYP per pupil expenditures were 
associated with slightly increased chances of being retained. 
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Table 2.24. Descriptive Statistics for Student Retention Data 


Variable Name 

N 

Mean 

SD 

Minimum 

Maximum 

Cohort 1 (1999-00) Student-Level Descriptive Statistics 

Grade group in 1999-00 

70,193 

0.42 

0.49 

0.00 

1.00 

Gender 

70,193 

0.55 

0.50 

0.00 

1.00 

Minority 

70,193 

0.82 

0.39 

0.00 

1.00 

Economically disadvantaged 

70,193 

0.77 

0.42 

0.00 

1.00 

Prior retention 

70,193 

0.12 

0.33 

0.00 

1.00 

Retention in 2000 

70,193 

0.05 

0.21 

0.00 

1.00 

Combined 2000 TAAS score 

70,193 

69.01 

16.28 

0.50 

97.00 

Days taught in OEYP 

70,193 

15.39 

8.29 

0.10 

30.00 

School attendance 

70,193 

96.13 

3.99 

32.60 

100.00 

OEYP type 

70,193 

0.28 

0.45 

0.00 

1.00 

OEYP days taught X OEYP type 

70,193 

2.00 

4.71 

0.00 

30.00 

Cohort 1 (1999-00) District-Level Descriptive Statistics 

Economically disadvantaged 

526 

0.74 

0.16 

0.08 

1.00 

Reading and math achievement 

526 

69.49 

5.74 

52.16 

90.70 

OEYP expenditure per pupil 

526 

284.97 

140.13 

18.16 

698.94 

Cohort 2 (2000-01) Student-Level Descriptive Statistics 

Grade group in 2000-01 

69,348 

0.42 

0.49 

0.00 

1.00 

Gender 

69,348 

0.56 

0.50 

0.00 

1.00 

Minority 

69,348 

0.81 

0.39 

0.00 

1.00 

Economically disadvantaged 

69,348 

0.77 

0.42 

0.00 

1.00 

Prior retention 

69,348 

0.13 

0.34 

0.00 

1.00 

Retention in 2001 

69,348 

0.05 

0.22 

0.00 

1.00 

Combined 2001 TAAS score 

69,348 

71.06 

15.50 

1.50 

97.00 

Days taught in OEYP 

69,348 

13.96 

7.58 

1.00 

30.00 

School attendance 

69,348 

95.85 

4.23 

31.82 

100.00 

OEYP type 

69,348 

0.33 

0.47 

0.00 

1.00 

OEYP days taught X OEYP type 

69,348 

2.16 

4.20 

0.00 

30.00 

Cohort 2 (2000-01) District-Level Descriptive Statistics 

Economically disadvantaged 

499 

0.75 

0.15 

0.08 

1.00 

Reading and math achievement 

499 

71.21 

6.69 

12.51 

89.13 

OEYP expenditure per pupil 

499 

308.79 

159.31 

52.59 

693.39 
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Table 2.25. HGLM Analyses of Retention Status of Cohort 1 
(1999-00) and Cohort 2 (2000-01) OEYP Students 


Conditional Model 

Cohort 1 

Cohort 2 

Student Level Variables 

District Level Variables 

Gamma 

Coefficient/ 

it) 

Gamma 

Coefficient/ 

it) 

Intercept 


-3.58 

(-60.71**) 

-3.48 

(-60.41**) 


Economically 
disadvantaged context 

-0.30 

(-0.72) 

0.64 

(1.47) 


Achievement context 

0.02 

(1.39) 

0.01 

(1.57) 


OEYP per pupil 
expenditure 

0.0017 

(4.20**) 

0.0022 

(6.15**) 

Gender 


0.34 

(8.74**) 

0.27 

(7.14**) 

Economically 

disadvantaged 


0.18 

(3.47*) 

0.24 

(4.55**) 

Prior retention 


-0.51 

(-8.24**) 

-0.52 

(-8.95**) 

School attendance 


-0.07 

(-19.13**) 

-0.05 

(-14.51**) 

Combined TAAS reading 
and math score 


-0.03 

(-33.25**) 

-0.03 

(-32.68**) 

Days taught in OEYP 


-0.06 

(-18.14**) 

-0.07 

(-19.50**) 

OEYP type 


-1.42 

(-11.65**) 

-1.55 

(-13.33**) 

Minority 


0.02 

(0.39) 

-0.00 

(-0.02) 

Grade group in OEYP 
year 


-0.19 

(-4.57**) 

-0.65 

(-14.98**) 

OEYP days taught X 
OEYP type 


0.08 

(7.82**) 

0.07 

(5.95**) 


Notes. All continuous variables without a true 0 were grand-mean centered. 
Student-level predictors were specified as fixed with error terms constrained to be 0. 
Intercepts varied randomly. 

*/?< . 01 ; **/?< . 001 . 
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CONCLUSIONS AND IMPLICATIONS 


First established by the 73rd Texas Legislature in 1993, the Optional Extended Year Program (OEYP) is a 
state-funded program with the goal of meeting the needs of elementary and middle school students who 
are at risk of not being promoted to the next grade level. Eunds allow districts to provide an extended-year 
program for up to 30 instructional days for eligible students, with the ultimate goal of reducing retention 
rates. School districts in which at least 35% of students in kindergarten through grade eight are from 
economically disadvantaged families are eligible for non-competitive grants to serve students in the 
OEYP. Districts’ level of funding is based on the amount necessary to offer extended-year services to not 
more than 10% of the at-risk student population in kindergarten through grade eight. The purpose of this 
study is to assess the effective use of OEYP funds to improve student learning for targeted students. The 
study encompasses four school years (1999-00 to 2002-03). 


What are the characteristics of districts and students receiving OEYP grants? 


> Characteristics of districts. Between 1999-2000 and 2002-03, the total number of districts 
receiving OEYP funds was 695, 682, 672, and 684, respectively. OEYP grantees were distributed 
across ah of the Education Service Center (ESC) regions. Nearly half of OEYP districts fall into the 
very small category. However, compared to the state, fewer OEYP districts are very small (fewer than 
1,000 students) and more are small and mid-sized (1,001 to 10,000 students). 

> Awards paid. The average award actually paid to districts was between $76,000 and $80,000. Paid 
awards ranged from $317 to over $5,000,000. Approximately 40% of districts received less than 
$10,000 for their OEYP program. 

^ Characteristics of students. About 190,000 students from nearly 700 school districts participated 
in the OEYP each year. OEYP participants are distributed across grades 1 through 8, with the largest 
proportion being third graders. This trend likely reflects the statewide commitment that students must 
achieve grade-level standards by third grade in order to be promoted (i.e.. Student Success Initiative). 

Compared to the state, OEYP served a greater proportion of Hispanic students (about 64% versus 
40% statewide), slightly more African-American students (about 18% versus 14%), and substantially 
less White students (about 17% versus 42%). OEYP students were also more likely to be 
economically disadvantaged (about 79% compared with 56% across Texas) and limited English 
proficient (about 31% compared with 15%). 


How did grant resources supplement existing educational programs? 


> OEYP program types. About two-thirds of OEYP students participated in an extended-year or 
intercession program only, whereas about one-fourth only participated in an extended-day program. 
Across four school years, the prevalence of extended-day and extended-week programs increased, 
while the emphasis on extended-year or intersession programs decreased. 

> Grade levels and content areas targeted. At least one third of the districts provided programs for 
students in grades 1 and 2, whereas nearly half targeted each of the grades 3 through 8. In district 
grant applications, mathematics and reading were the most frequently targeted content areas for 
OEYP programs. 

> Program activities. OEYP instructional activities focused most often on reading/language arts (e.g., 
reading strategies, computer-assisted instruction, TEKS) and mathematics (e.g., computer-assisted 
instruction, TEKS). Districts mainly focused their professional development opportunities for 
teachers and staff on instructional strategies and strategies for teaching students at risk. Districts most 
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frequently planned to involve parents through conferences, parent workshops, and various 
communication strategies. 

> Instructional days available. On average, the number of OEYP instructional days available for 
students declined across years, with 20 days in 1999-00, 17 in 2000-01, and 15 in both 2001-02 and 
2002-03. Intercessions or extended-year programs had the largest number of instructional days 
(between 19 and 21 days each year). 


What was the effect of grant resources on targeted students? 


To determine the effectiveness of the OEYP program in improving student success, we examined OEYP 
students’ attendance and retention rates and performance on state-level assessments. In addition, we 
explored the relationships between student achievement and extended OEYP participation, program 
types, and instructional days. 

> Attendance. Students’ average OEYP attendance rates for four school years (81% to 86%) are 
considerably lower than their attendance rates during the regular school year (about 96%). OEYP 
students’ attendance rates during the regular school year are comparable to state averages and 
remained relatively stable across years. Students attended OEYP extended-year and intercession 
programs at a higher rate (from 86% to 90% of instructional days) than extended-day (54% to 80% of 
days) or extended-week (62% to 70% of days) programs. It is possible that districts may have 
instituted more stringent attendance requirements for extended-year and intercessions programs. 

> Retention. Eindings suggest that districts are more likely to use retention in the early grades as a 
means to support student academic performance. Compared to state averages, retention rates of 
OEYP students are far higher for grades 1 through 3 (about 17, 12, and 6 percentage points, 
respectively) but only slightly higher for students in grades 4 through 8 (about 2 percentage points). 
Across four years, about 23% of OEYP first graders, 16% of second graders, and 9% of third graders 
were retained. In contrast, retention rates for students in grades 4 through 8 were typically less than 
5%. Eor all grade levels, student retention rates tended to increase across the four OEYP program 
years. 

^ State-level assessments. Passing rates on state assessments (TAAS reading, math, writing, 
science, social studies, and all tests) were well below state averages for the four OEYP student 
cohorts studied. However, for cohort 1 (1999-2000) and cohort 2 (2000-01) students, TAAS passing 
rate gains (from the year before to the year after full OEYP participation) exceeded state gains. On 
the other hand, the TAAS to TAKS passing rate gains for cohort 3 students (2001-02) were mostly 
less than state gains. Thus, the achievement gap between OEYP students and state averages was 
narrowed for cohorts 1 and 2, but not for cohort 3. 

> Extended participation opportunities and achievement. We also investigated the tendency for 
students to participate in the OEYP for more than one year. About 60% of OEYP students served in 
1999-00 only participated that year, while about 40% participated at least one more year. With a few 
exceptions, cohort 1 students (1999-00) who participated in OEYP for more years had lower TAAS 
passing rates than their counterparts who attended fewer years. Thus, not surprisingly, it is the lower 
performing students who are more likely to participate in OEYP for multiple years. 

> OEYP program type and student achievement. It was also of interest to determine whether 
students’ performance varied according to the type of OEYP program they attended. Clearly, students 
who only participated in extended-day or extended- week programs had higher TAAS and TAKS 
scores and lower retention rates than students enrolled in extended year or intercession programs. 

Still, this does not suggest that there is a causal link between extended-day and extended-week 
programs and improved academic performance. Instead, it is plausible that lower achieving students 
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may be required to attend summer school (extended year) or intercessions (year-round schools) to 
avoid retention. 

> Instructional days and student achievement. In general, it appears that a students’ academic 
need may dictate the number of available OEYP instructional days. In the year following full OEYP 
participation, higher TAAS and TAKS passing rates are associated with lower percentages of OEYP 
prescribed instructional days attended. Eor example, 72% of students in cohort 1 (1999-00) who 
attended 0-49% of OEYP days passed TAAS Reading, whereas only 66% of students who attended 
90-100% of days passed TAAS Reading. 


What program elements are associated with student outcomes? 


To further explore the association between OEYP student and district characteristics and TAAS reading 
and mathematics TEI scores, researchers used hierarchical linear modeling (HEM). Separate HEM 
analyses were conducted using OEYP participants in 1999-00 (cohort 1) and 2000-01 (cohort 2). Each 
group included more than 40,000 students from over 400 school districts. These students participated in 
extended-day or extended-year/intercession programs in their respective districts. Separate HEM analyses 
were also conducted for retention. Each group included about 70,000 students from about 500 school 
districts. 

Student-Level Factors and Outcomes 

> Controlling for important student-level characteristics (i.e., academic and social background), 
extended-day participants had higher TAAS reading and mathematics scores than 
extended-year/intercession participants. This suggests that students who receive tutorial and other 
assistance at the “point of need” during the school year may do better academically than those who 
attend an intercession or summer school after they have failed to master content during a grading 
period or year. 

> There was no positive relationship between the instructional days students spend in OEYP (up to 30 
instructional days) and TAAS scores. However, for otherwise similar students, more instructional 
days in OEYP decreased the chances of retention for extended- year/intercession participants. This 
may be explained by the fact that OEYP extended-year/intercession programs have a fixed number of 
scheduled days of instruction for students who are in danger of being retained, and successful 
program completion precludes retention. In contrast, for extended-day participants, fewer 
instructional days in OEYP decreased chances of retention. Overall findings for extended-day 
programs suggest that students may receive homework assistance, tutorials, and remedial study on an 
as-needed basis. Thus, students having less of a need for such help may attend fewer days. 

> A student’s school attendance rate was an important predictor of academic performance. School 
attendance had a stronger influence on TAAS mathematics scores than on TAAS reading scores. In 
addition, for otherwise similar students, an increase in the school attendance rate decreased the 
chances of retention. Student attendance is important because it provides an indicator of student 
overall commitment to and engagement in school. Consistent attendance also helps students to “keep 
up” with assignments and participate in class activities. This is especially important for mathematics, 
which has a spiraling scope and sequence, with lower level knowledge and skills prerequisites for 
mastering higher level concepts. 

District-Level Factors and Outcomes 

> After controlling for student-level characteristics, OEYP students’ academic achievement and chance 
of retention varied significantly by district. This suggests that some districts and schools are more 
successful in meeting the needs of students in at-risk situations. 


34 




> Evidence from this study confirms the importance of the school context. Other student-level factors 
being equal and net of district social context and OEYP expenditures, OEYP students having higher 
achieving classmates performed better in TAAS reading and mathematics. 

> Equally important, there was no compelling evidence that increasing per-pupil OEYP expenditures 
improved student performance. There was no significant relationship between OEYP dollars spent per 
pupil and TAAS reading and mathematics scores. Moreover, higher OEYP per-pupil expenditures 
were associated with a slightly increased chance of student retention. Results regarding expenditures 
suggest that how districts use available resources is critically important in improving outcomes for 
students at risk. Simply increasing available dollar allocations may not achieved desired effects. 


What are the implications for addressing the needs of students at risk? 


> Enhancing the academic prospects for students at risk hinges on overaii improvement of 
iearning opportunities in schoois and ciassrooms. Eindings from this study reinforce the 
importance of improving the overall school environment as a means to enhance the learning 
opportunities of students at risk. Results for “value-added” modeling using HEM suggest that, after 
controlling for important student-level characteristics, some districts and schools are more successful 
than others in supporting the academic performance of students at risk. Eurthermore, evidence 
indicates that OEYP students who attend schools with higher achieving classmates performed better 
on state-level assessments. Results for this study are consistent with other research citing the 
importance of the school context. Research shows that students tend to learn more when taught by 
“effective teachers” and teachers become more effective when they work in “effective schools” 
(Stringfield & Datnow, 2002). Waxman, who has studied at-risk populations extensively, says, “It is 
apparent that some of the risks associated with students’ failure in school are due to the particular 
school the student attends” (1992, p. 7). Similarly, others suggest that we have used remediation as a 
means to “change the student” rather than altering the student- school relationships through 
educational change (Bitting, Cordero, & Baptiste, 1992). 

> Efforts directed at improving student attendance during the reguiar schooi year may have a 
greater effect on student achievement than remediai interventions. Results for this study 
reinforce the importance of school attendance in the academic success of students in at-risk 
environments. In this study, school attendance was an important predictor of performance on state- 
level assessments, especially mathematics. An increase in school attendance was also associated with 
decreased chances of retention. Bindings suggest that resources directed toward addressing the factors 
that reduce school attendance during the regular school year may produce greater academic gains for 
students at risk than assigning them to remedial interventions with other low-achieving students. 

> Low student attendance in extended-day, -week, and -year programs limits program 
effectiveness. Consistent student participation in OEYP initiatives was a problem. Bindings for four 
student cohorts (2000 to 2003) suggest that student attendance in programs is sporadic. Eor example, 
students enrolled in extended year/intercession programs attended about 89% of days. This meant that 
in a typical 20-day program, a student would receive about 17.5 days of instruction. Attendance was 
even less regular for extended-day and extended-year programs. A typical student enrolled only in an 
extended-day program attended about 80% of days. Thus, an average student in cohort 4 would have 
attended 5.6 of 7.4 after-school program days. It is doubtful that the number of days available and 
attended is adequate to impact either achievement or retention. In an extensive study of the allocation 
and duration of schooling. Glass (2002) found that small increases (10-15%) in time for schooling 
resulted in no significant achievement gains. 
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> Little is known about the quality of programs funded by OEYP. As noted above, quantitative 
evidence indicates that the number of available OEYP instructional days (which declined from 20 to 
15 over four grant years) may be inadequate to make a difference in achievement for at-risk students. 
In addition, we know very little about the quality of the learning experiences available for students. 
Based on a review of a sample of district proposals (2002-03), it appears that OEYP programs focus 
primarily on reading and mathematics and many districts use computer-assisted programs to deliver 
instruction (usually learning systems for basic skill acquisition). Beyond this, there is little available 
evidence to show whether the opportunities to learn in OEYP programs were appropriate for students 
in at-risk situations. 

> Student retention rates increased between 1999-00 and 2002-03, especially for first, second, 
and third graders. If we measure the OEYP against its primary goal — the reduction of student 
retention — outcomes suggest that the program was unsuccessful in attaining its overarching goal. 
Retention rates for students at risk increased across four years. Districts increasingly retained students 
in first grade (from 18% to 26%), second grade (from 12% to 17%), and third grade (from 7% to 
11%). Retention rates also increased about 1 percentage point for grades 4 to 8. Increased retention of 
at-risk students in the early grades is almost certainly a response to the Student Success Initiative and 
the requirement that students meet grade-level standards. However, further research is needed to 
examine the impact of increased retention in light of the fact that other studies have shown 
detrimental effects of retention on students (e.g., Nagaoka & Roderick, 2004). 

^ The cost-effectiveness of the OEYP is questionable. Efforts aimed at examining associations 
between OEYP funding levels and both student achievement and retention suggest there was no 
significant relationship between OEYP dollars spent per pupil and academic achievement or reduced 
retention. Our study was not designed to be a comprehensive cost-benefit analysis, but findings raise 
questions about the cost-effectiveness of the initiative statewide. 

> State-level initiatives aimed at improving instruction and learning for students at risk should 
be accompanied by evaluations to study program effectiveness. Discretionary grants provide a 
means for states to gather data and determine future policies regarding funding for programs and 
services for students at risk. However, conducting scientifically rigorous evaluations relies on 
designing and conducting studies at the onset of funding and program implementation. Eunding for 
future initiatives supporting students at risk should be accompanied by resources for program 
evaluations. 
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Chapter 3 

TEXAS AFTER SCHOOL INITIATIVE (TASI) 


INTRODUCTION 

In 1999, the Texas Legislature appropriated $10.5 million for the creation of the Texas After School 
Initiative (TASI). The program funds after-school programs targeting at-risk middle school students. 
TASI services are intended for students ages 10 to 14 at-risk of academic failure and/or at-risk of 
committing juvenile offenses. According to TASI standards, identification of students at-risk of academic 
failure is based on meeting any of these characteristics: 

• Failure to pass one or more TAAS suhtests at the last administration period, 

• Retention in one or more grade levels, 

• Failing one or more classes in a reporting period, 

• Recurrent pattern of absences, 

• Counselor and special program referrals with appropriate documentation, 

• Teacher or principal observations and documentation, 

• Parent request and principal approval. 

Characteristics of students at-risk of committing juvenile offenses include: 

• Referrals to the Juvenile Justice System, 

• Recurrent pattern of absences, 

• Disciplinary referrals with documentation from counselors and special program staff, 

• Teacher or principal observations and documentation, 

• Parent request and principal approval. 

TASI programs must address the needs of targeted students through an academics-based curriculum 
linked to the TEKS, a character education or citizenship component, and parent and/or mentor 
involvement. The purposes of TASI programs include increasing academic achievement, decreasing 
referrals to the Juvenile Justice System, and obtaining parent and mentor involvement in activities 
targeting at-risk middle school students. The 77th Texas Legislature renewed TASFs $10.5 million 
appropriation for the 2002-03 biennium. 

Organization of the Chapter 

Sections to follow include a brief literature review of after-school programs and an overview of this 
study’s methodology. Additionally, TASI program findings are presented for the following topics: 

(a) TASI grantees and their students; (b) TASI program activities; (c) student outcomes for core courses, 
discipline referrals, attendance, state-level assessments, and retention; (d) association of student- and 
district-level factors with academic achievement; (e) association of student- and district-level factors with 
retention; and (f) conclusions and implications. 

RESEARCH ON AFTER-SCHOOL PROGRAMS 

After-school programs are not new on the educational landscape. Since the late 1800s, after-school 
programs and clubs have existed with the original goal of keeping youngsters off the streets where they 
are unsupervised and susceptible to the dangers of urban life (Halpern, 2002). More recently, as the 
emphasis on standardized test scores and accountability has increased, educators have begun to view 
after-school programs as an opportunity to enhance student achievement by extending academic activities 
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into the hours heyond the typical school day. Surprisingly, few large-scale experimental or quasi- 
experimental research studies have considered the effect of after-school programs on either academic 
achievement or social and developmental measures. Research on after-school programs has more often 
relied on less rigorous research designs, thus limiting conclusions. Another challenge associated with 
studying after-school programs is the varied configurations and objectives that exist. Some programs 
focus on providing recreational and cultural activities, others target prevention of risky behaviors among 
youth (e.g., alcohol/drug use, gang activity), and still others have primarily academic objectives. 

The available research on after-school programs generally shows mixed results. A number of studies have 
shown positive impacts of after-school programs on non-academic measures such as reductions in risky 
behavior and increases in positive peer relationships and conflict resolution (Beuhring, Blum, & Rinehart, 
2000; Pierce, Hamm, & Vandell, 1999; Zill, Nord, &Loomis, 1995 as cited in Miller 2001). Several 
quasi-experimental studies conducted to evaluate after-school programs in school districts have indicated 
positive impacts on standardized test scores. A 1998 evaluation of Sacramento’s START program showed 
that 67% of participants improved their reading and math scores as compared to the matched-comparison 
group students (Lamare, 1998 as cited in Brown, McComb, & Scott-Little, 2003). Similarly, an 
evaluation of four After-School Learning and Safe Neighborhoods Partnerships Programs (ALSSNPP) in 
Santa Ana Unified School District found that participants’ reading and math scores improved 
significantly over matched-comparison group students, especially for students who attended regularly or 
were classified as Limited English Proficient (Prenovost, 2001 as cited in Brown, McComb, & Scott- 
Little, 2003). 

The largest- scale study of after- school programs to date — the national evaluation of the 21st Century 
Learning Community Learning Centers — showed much less promising results. This program, first 
authorized by Congress in 1994, established after-school centers in about 7,500 public schools across the 
nation to offer academic and recreational activities. First- year evaluation results indicate that 21st Century 
centers have limited academic impact on students. Test scores and grades in most subjects for center 
participants were virtually the same as comparison group students. Additionally, participation in 21st 
Century programs did not impact students’ rate of homework completion. Subgroup analysis, however, 
did show some small, positive effects on grades, absenteeism, and tardiness for African American and 
Hispanic middle school students (Dynarski, Moore, Mullens, Gleason, James-Burdumy, Rosenberg, 
Pistorino, Silva, Deke, Mansfield, Heaviside, & Levy, 2003). This finding supports other research 
showing that minority and low-income students reap greater benefits from after-school programs (Miller, 
2001; Hamilton & Klein, 1998; Schinke, Cole, & Poulin, 1998; Witt, 1997). 

Findings for possible non-academic impacts of 21st Century programs were equally mixed. Programs 
helped increase the proportion of students being cared for by an adult after school by reducing the 
percentage cared for by an older sibling, but did not reduce the number of students staying at home alone 
after school (“latchkey” kids). As for social and behavioral outcomes, participation in 21st Century 
programs had virtually no impact. In fact, middle school participants were more likely to report selling or 
using drugs than comparison group students (although the incidence was low). Furthermore, social and 
developmental skills, such as conflict resolution and teamwork were not impacted by the program 
(Dynarski, et ah, 2003). In contrast, participation in 21st Century programs was associated with increased 
parental involvement in the school. Parents of both elementary and middle school participants were more 
likely to volunteer at their child’s school, attend meetings, or help their child with homework (Dynarski, 
et al., 2003). 

Overall, research has shown some positive impacts of after-school programs on academic and non- 
academic measures; however, the small-scale nature and non-experimental design of many of these 
studies limit their generalizability. In contrast, the national evaluation of the 21st Century Fearning 
Community Fearning Centers indicated few positive academic or non-academic impacts. Most 
researchers agree that one of the primary challenges in studying after-school programs is low attendance 
rates on the part of students. Most studies, including the 21st Century evaluation, have found that student 
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participation in after-school programs is limited and sporadic. Student attendance in 21st Century 
programs averaged less than two days a week (Dynarski, et ah, 2003). Therefore, student attendance 
issues, if not addressed, will continue to serve as a challenge to widespread academic and social benefits 
of after-school programs. 


METHODOLOGY 

The purpose of this evaluation is to assess the effective use of TASI funds to create after-school programs 
supporting the academic success of targeted students in grades 6 through 8. This report relies on a 
combination of quantitative and qualitative data and analysis methods as described below. The study 
includes 60 school districts and 194 campuses receiving both original and continuation TASI funding. 
Analyses examine results for original grant dollars used over three semesters (spring 2000 through spring 
2001) and continuation funding over four semesters (fall 2001 through spring 2003). During four school 
years (1999-00, 2000-01, 2001-02, and 2002-03), approximately 100,000 unique students participated in 
TASI. Student counts reflect enrollment figures that use estimates to account for missing data (see 
Table 3.3). 

Data Sources 

The primary data sources include TASI program and activity reports submitted by districts and student 
demographic and performance data collected from the Texas Education Agency (TEA). 

Program and activity reports. TASI grant recipients were required to submit program and activity 
reports to the TEA after each semester in which they served targeted students. Across four program years 
(1999-00 through 2002-03), districts were asked to submit two reports after each of seven semesters in 
which they served students. The program report requested district-level information, such as general 
program information; activities supporting the academics-based, character/citizenship, and parent/mentor 
involvement components of TASI; staff participation and volunteer involvement; professional 
development; information dissemination; and district opinions regarding the most successful components. 
The procedures used to collect data on program reports was altered during spring 2003, and many data 
elements available in earlier terms were not collected during this term. As a result, some tables include 
data for only one semester of the final cohort (2002-03). 

In addition to the program report, districts also submitted an activity report each semester with student- 
level data. Activity reports provided information for each TASI program participant in seven areas: 
student demographics, student eligibility, retention, program attendance, additional activities, discipline 
referrals, and student performance. As with the program report, all information is self-reported. During 
the course of the seven program terms, report format changes resulted in some data discontinuity. 

Student demographic and performance data. Researchers gathered other student-level data from the 
Texas Public Education Information Management System (PEIMS) and the Texas Academic Excellence 
Indicator System (AEIS). Student-level data supplied by participating school districts’ activity reports 
were matched to PEIMS and AEIS data to create a set of master databases. Elements in the databases 
included student demographic information, such as ethnicity, gender, EEP status, and grade level; Texas 
Assessment of Academic Skills (TAAS) and Texas Assessment of Knowledge and Skills (TAKS) scores, 
and attendance and promotion rates. 

Limitations 

Data accuracy. District compliance with reporting requirements varied over the duration of the initiative 
(see Table 3.1). Overall, districts submitted program reports at a lower rate than activity reports. 
Submission rates for program reports varied from 23% (fall 2002) to 67% (spring 2000). Eor activity 
reports, the first two terms (spring and fall 2000) had the lowest report submission rates, 62% and 65%, 
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respectively. Beginning with the spring 2001 term, TEA staff contacted districts about report submission. 
As a result, the submission rate increased slightly for the remaining terms. 


Table 3.1. Districts Submitting Program and Activity Reports 


Term 

Program Reports 
(District Level) 

Activity Reports 
(Student Level) 

N 

% 

N 

% 

Spring 2000 

40 

67 

37 

62 

Fall 2000 

16 

27 

39 

65 

Spring 2001 

23 

38 

42 

70 

Fall 2001 

16 

27 

46 

77 

Spring 2002 

16 

27 

44 

73 

Fall 2002 

14 

23 

51 

85 

Spring 2003 

26 

43 

47 

78 


N=60 participating districts. 


Varying report submission rates are problematic because, first, it is possible that the districts included in 
the analysis, by virtue of submitting reports, are not representative of all the districts funded under TASI. 
Secondly, when comparing results from year to year, it is possible that different response rates across 
terms lead to incorrect conclusions, as the districts are not the same from one term to the next. As 
Table 3.2 illustrates, of the 60 original and continuing districts, only 30% of districts submitted activity 
reports for all seven terms, but 80% of districts submitted reports for four or more terms. This group of 
48 districts forms the core TASI group included in analyses — thus, results are most representative of these 
districts rather than the districts that submitted a report only once or twice or never submitted a report. 


Table 3.2. Number of Student Activity Reports Submitted by Districts 


Number of Reports 
Submitted 

Number of 
Districts 

Percent of 
Districts 

Cumulative 

Percent 

Seven terms 

18 

30.0 

30.0 

Six terms 

10 

16.7 

46.7 

Five terms 

14 

23.3 

70.0 

Four terms 

6 

10.0 

80.0 

Three terms 

6 

10.0 

90.0 

Two terms 

2 

3.3 

93.3 

One term 

3 

5.0 

98.3 

Never submitted a report 

1 

1.7 

100.0 

Total 

60 

100.0 



Estimating student participation. Student data for each term were combined to estimate the total 
number of unique students served during the four-year grant period (see Table 3.3). By replacing missing 
term data with a weighted district term average, the number of unique students served during four TASI 
program years was estimated at 101,771 students. (See Appendix B for a full explanation.). The estimated 
number of students for each program year ranged from 13,313 (spring 2000) to 31,945 (2002-03). 
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Table 3.3. Number of Students Served in TASI by School Year 


Year 

Term 

Duplicated Student Count 

Unique Student Count 

Activity Report 
N 

Estimated N 

Activity Report 
N 

Estimated N 

1999-00 

Spring 2000 

9,395 

13,313 

9,395 

13,313 

2000-01 

Fall 2000 

11,142 

17,462 

15,173 

24,742 

Spring 2001 

11,207 

16,230 

2001-02 

Fall 2001 

13,246 

18,801 

21,726 

31,771 

Spring 2002 

16,512 

21,612 

2002-03 

Fall 2002 

19,987 

21,579 

26,437 

31,945 

Spring 2003 

17,235 

21,535 


Total 

98,724 

130,532 

72,731 

101,771 


TASI GRANTEES AND THEIR STUDENTS 
Location of Districts Receiving Grants 

> The distribution of TASI grantees generally corresponds to Texas population centers. 

The distribution of TASI grantees, in most cases, corresponds with Texas population regions. However, 
densely populated regions (e.g., ESC 4, Houston and ESC 10, Richardson/Dallas) and the Rio Grande 
Valley region (ESC 1, Edinburg) have more programs (Table 3.4). 


Table 3.4. TASI Grantees by ESC Region 


ESC 

ESC 

Number of 

Percent of 

Statewide ESC 

Region 

Location 

Grantees 

Grantees 

Distribution 

1 

Edinburg 

6 

10.0 

4.1 

2 

Corpus Christi 

4 

6.7 

3.9 

3 

Victoria 

2 

3.3 

3.3 

4 

Houston 

15 

25.0 

8.3 

5 

Beaumont 

1 

1.7 

2.9 

6 

Huntsville 

1 

1.7 

4.9 

7 

Kilgore 

2 

3.3 

8.4 

8 

Mt. Pleasant 

0 

0.0 

3.9 

9 

Wichita Falls 

0 

0.0 

3.3 

10 

Richardson 

7 

11.7 

9.2 

11 

Ft. Worth 

4 

6.7 

7.2 

12 

Waco 

2 

3.3 

7.0 

13 

Austin 

4 

6.7 

5.8 

14 

Abilene 

0 

0.0 

3.6 

15 

San Angelo 

1 

1.7 

3.6 

16 

Amarillo 

1 

1.7 

5.3 

17 

Lubbock 

1 

1.7 

5.1 

18 

Midland 

1 

1.7 

2.9 

19 

El Paso 

3 

5.0 

1.3 

20 

San Antonio 

5 

8.3 

6.0 


Note. A total of 60 districts received both original and continuation funding. 


41 




Characteristics of Districts Receiving Grants 

> TASI programs are more heavily concentrated in larger districts (more than 10,000 students). 
Smaller districts seldom had TASI programs. 

TASI programs tend to be more heavily concentrated in larger districts — 62% fall into the large or very 
large categories. An additional 23% are midsize. Only one very small district had a TASI program. In 
contrast, larger percentages of districts statewide are in the very small or small categories (Table 3.5). 


Table 3.5. TASI Grantees by District and Campus Student 
Enrollment Size Categories 


District Size: Student Enrollment 

Number 

Percent 

State 

Very small: Eewer than 1,000 

1 

1.7 


Small: 1,000 to 3,000 

8 

13.3 


Midsize: 3,001 to 10,000 

14 

23.3 

12.9 

Large: 10,001 to 25,000 

16 

26.7 

3.8 

Very large: More than 25,000 

21 

35.0 

3.2 


Source. TEA AEIS database 2002-03. 


> Compared to the state, TASI districts and campuses have larger proportions of Hispanic, LEP, 
and economically disadvantaged students, smaller proportions of White students, and nearly 
equal percentages of African Americans. 

As Table 3.6 shows, TASI districts and campuses enroll larger proportions of Hispanic students (51- 
54%), smaller proportions of White students (23-28%), and comparable percentages of African American 
students (18%) compared to state averages. Additionally, TASI districts and campuses have larger 
percentages of economically disadvantaged (about 60%) and Limited English Proficient (LEP) students 
( 12 - 22 %). 


Table 3.6. TASI District and Campus Student Demographics (Percent) 


Student Demographics 

TASI 

Districts 

TASI 

Campuses 

State MS 
Average 

White 

23.4 

27.9 

41.9 

Hispanic 

54.4 

51.3 

40.6 

African American 

18.3 

17.6 

14.4 

Economically disadvantaged 

61.5 

59.2 

50.1 

Limited English proficient 

21.9 

11.9 

8.3 

Special education 

10.7 

13.4 

13.2 


Source. TEA AEIS database 2002-03. 


Characteristics of Students Participating in TASI 

> Based on estimated counts, the number of students served in TASI programs increased 
during the four years of the grant from about 13,000 in spring 2000 to almost 32,000 in 
2002-03. 

Only 13,313 students participated in TASI during the “start-up” semester (spring 2000). However, the 
number of student participants increased steadily thereafter: 24,742 in 2000-01; 31,771 in 2001-02; and 
31,945 in 2002-03. At the same time, the mean number of students served per district in TASI programs 
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increased from 254 (spring 2000) to 481 (2002-03). TASI participants included a nearly equal proportion 
of sixth, seventh, and eighth graders (Table 3.7). 


Table 3.7. TASI Program Participation 


Number 

Spr 2000 

2000-01 

2001-02 

2002-03 

Districts 

37 

47 

53 

55 

Campuses 

107 

146 

172 

194 

Total students 

9,395 

15,173 

21,726 

26,437 

Grade 6 students 

35% 

31% 

34% 

29% 

Grade 7 students 

36% 

37% 

34% 

38% 

Grade 8 students 

29% 

32% 

32% 

33% 

Students per district (mean) 

254 

323 

405 

481 

Students per campus (mean) 

88 

104 

126 

136 

Estimated student population^ 

13,313 

24,742 

31,771 

31,945 


“Estimated student population based on extrapolation from known data (see Appendix B). 
Source: District activity reports. 


> More than three-fourths of TASI students are Hispanic (about 53%) and African American 
( 22 %). 

Just over half of TASI participants are Hispanic, while approximately one-fifth are African American and 
one-fifth are White. Male and female students are equally represented among TASI participants 
(Table 3.8). 


Table 3.8. Demographic Characteristics of TASI Participants 


Student Demographics 

Spr 2000 

2000-01 

2001-02 

2002-03 

White 

21% 

24% 

22% 

17% 

Hispanic 

54% 

53% 

56% 

57% 

African American 

24% 

20% 

20% 

24% 

Eemale 

48% 

48% 

49% 

50% 

Male 

52% 

52% 

51% 

50% 


Source: District activity reports and PEIMS. 


> TASI programs typically offered about four instructional days per week, with nearly two 
program hours each day. 

On average, students participated in TASI programs slightly less than 4 instructional days per week (3.5 
to 3.9) and about 2 hours per day. The mean duration for TASI programs ranged between 13.4 and 15.6 
weeks per semester. The mean number of programming days varied across years, from 48.9 in spring 
2000 to 58.3 during the 2001-02 school year (Table 3.9). 


Table 3.9. General TASI Program Information 


Program Characteristics 

Spr 2000 

2000-01 

2001-02 

Fall 2002 

Program days per week (mean) 

3.7 

3.5 

3.8 

3.9 

Program hours per day (mean) 

2.2 

1.9 

1.9 

2.0 

Number of program weeks (mean) 

13.4 

15.6 

15.3 

14.2 

Number of program days (mean) 

48.9 

56.0 

58.3 

55.6 


Source: District program reports. 
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> TASI programs targeted students at risk of failure — either those meeting the state 
Compensatory Education requirements or meeting other risk factors. 

While TASI grantees could provide program services for students having no risk characteristics, these 
students could comprise no more than 35% of all participants served. Although student eligihility varied 
hy year, overall trends indicate that TASI programs served a large majority of at-risk students each school 
year (between 73% and 90%). Districts and campuses most frequently served students meeting state 
Compensatory Education requirements, although these percentages were higher in the first spring term 
(77%) compared to subsequent school years (47% to 54%). (See Table 3.10.) 


Table 3.10. Student Eligibility by Semester (Percent) 


Eligibility Category 

Spr 2000 

2000-01 

2001-02 

2002-03 

Meets state Compensatory 
Education requirements 

76.9 

46.5 

53.8 

48.0 

Meets other risk factors 

12.8 


25.0 

26.2 

No at-risk characteristics 

10.3 

21.2 

21.2 

25.8 


Note. Between 2.2% and 22.5% of students lacked eligibility data. 
Source: District activity reports. 


> Compared to White students, Hispanic and African American students were more likely to be 
eligible for TASI services under Compensatory Education requirements. 

Ethnic distributions varied by student eligibility categories. Earger percentages of Hispanic students met 
state Compensatory Education requirements or other risk factors. African American students more 
frequently met Compensatory Education requirements. In contrast, greater percentages of White students 
met other risk factors or no risk factors. TASI programs generally served comparable percentages of male 
and female students, although male students were more likely to be identified as at risk (Table 3.11). 


Table 3.11. Distribution of Student Gender and Race/Ethnicity by 
Semester and Student Eligibility (Percent) 


Student Demographics 

Spring 

2000 

2000-01 

2001-02 

2002-03 

Meets state Compensatory Education requirements 

Hispanic 

57.4 

57.8 

58.3 

61.2 

African American 

24.6 

22.0 

24.3 

26.1 

White 

16.5 

18.2 

15.1 

11.3 

Male 

52.1 

54.3 

52.1 

52.4 

Female 

47.9 

45.7 

47.9 

47.6 

Meets other risk factors 

Hispanic 

50.4 

52.1 

59.4 

64.5 

African American 

19.5 

17.9 

16.7 

18.7 

White 

28.5 

24.1 

22.6 

15.7 

Male 

56.1 

52.9 

53.1 

52.3 

Female 

43.9 

47.1 

46.9 

47.7 

No at-risk characteristics 

Hispanic 

36.2 

44.8 

42.6 

44.4 

African American 

24.5 

14.0 

15.5 

23.5 

White 

37.7 

34.9 

38.9 

29.1 

Male 

45.8 

48.1 

45.5 

43.6 

Female 

54.2 

51.9 

54.5 

56.4 


Note. Between 0.1% and 0.4% of students lacked gender data. Between 
0.6% and 2.5% of students lacked ethnicity data. 

Source: District activity reports. 
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> The majority of students served by TASI programs had never been retained in grade. 

Approximately 8% of TASI students had repeated one grade, and approximately 1% had multiple 
retentions. These rates are markedly higher than state-level retention statistics reported for sixth (about 
1.5%), seventh (about 2.7%), and eighth (about 1.9%) grades. Not surprisingly, participants meeting state 
Compensatory Education requirements or with other risk factors were more likely to have been retained 
in grade than students with no at-risk characteristics (Table 3.12). 


Table 3.12. Prior Student Retention by Semester and Student 
Eligibility (Percent) 


Retention Category 

Spring 

2000 

2000-01 

2001-02 

2002-03 

All TASI participants 

Never retained 

91.9 

91.4 

93.3 

88.6 

Retained in one grade 

7.4 

8.2 

6.3 


Retained in two or more grades 

0.7 

0.4 

0.3 


Meets state Compensatory Education requirements 

Never retained 

90.9 

89.2 

91.2 

85.6 

Retained in one grade 

8.4 

10.2 


13.1 

Retained in two or more grades 

0.8 

0.7 

0.5 

1.4 

Meets other risk factors 

Never retained 

86.6 

91.4 

92.0 

88.8 

Retained in one grade 

12.4 

8.1 

7.4 


Retained in two or more grades 

1.1 

0.5 

0.4 


No at-risk characteristics 

Never retained 

94.1 

97.6 

98.6 

93.7 

Retained in one grade 

5.7 

2.4 

1.4 


Retained in two or more grades 

0.3 

0.0 

0.0 



Source'. District activity reports. 


TASI PROGRAM ACTIVITIES 


Required Components 

TASI after-school programs for students of middle school age, 10 to 14 years old, were required to 
directly address the needs of students by incorporating three components: 

• an academic-based curriculum linked to the Texas Essential Knowledge and Skills (TEKS), 

• a character/citizenship education component, and 

• a plan for parental and/or mentor involvement. 

Each grantee designed a program that would help students gain practical knowledge and skills which 
would enable them to become successful in school as well as successful and productive adults in later life. 
As part of their program evaluations, districts identified strategies used to provide academic and 
character/citizenship services as well their plan for parent or mentor involvement. 

> TASI districts dedicated the greatest percentage of after-school time to the academic-based 
component. 

Eor more than three-quarters of TASI programs, 41% or more of after-school time was spent on 
academics. In contrast, the majority of programs spent 40% or less of time on the character/citizenship 
component and 20% or less time on the parental or mentor component (see Table 3.13 and Eigure 3.1). 
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Table 3.13. Time Dedicated to Various TASI Program Components 


Component 

Spr 2000 
A=40 

2000-01 

A=28 

2001-02 

A=21 

Fall 2002 
A=14 

Academics-based 

20% or less 

5.0 

7.7 

3.1 

7.1 

21-40% 

12.5 

7.7 

6.3 

7.1 

41-60% 

22.5 

28.2 

21.9 

28.6 

61-80% 

42.5 

38.5 

25.0 

28.6 

8 1 % or more 

17.5 

17.9 

43.8 

28.6 

Character/citizenship 

20% or less 

28.2 

43.6 

32.3 

42.9 

21-40% 

53.8 

30.8 

45.2 

50.0 

41-60% 

12.8 

15.4 

9.7 

7.1 

61-80% 

2.6 

5.1 

3.2 

— 

8 1 % or more 

2.6 

5.1 

9.7 

— 

Parent/mentor involvement 

20% or less 

52.5 

71.8 

62.5 

71.4 

21-40% 

42.5 

12.8 

9.4 

21.4 

41-60% 

2.5 

7.7 

15.6 

- 

61-80% 

2.5 

5.1 

3.1 

- 

8 1 % or more 

- 

2.6 

9.4 

7.1 



□ 20% or less 021-40% 041-60% 061-80% ■81%ormore 


Figure 3.1. Mean percent of districts dedicating various time allocations 
to TASI program components (four cohorts). 

Program Configuration 

> After-school programs most frequently relied on medium-sized group activities (7 to 19 
students) for both academic-based and character/citizenship program components, 

TASI programs used a variety of grouping configurations: however, medium-sized groups (with 7 to 19 
students) occurred most frequently. Individual and small-group activities (6 or less students) were also 
commonly used. Large-group activities (20 or more students) occurred suhstantially less often in TASI 
programs. Students, however, were more likely to participate in large-group character/citizenship 
activities compared to academic activities. (See Table 3.14 and Figure 3.2.) 
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Table 3.14. Types of Student-Level Services Provided - Academic and Character/Citizenship 
Components (Percent of Programs) 


Grouping Configurations 

Spring 

2000 

A=40 

2000-01 

A=28 

2001-02 

A=21 

Fall 

2002 

A=14 

AC 

CC 

AC 

CC 

AC 

CC 

AC 

CC 

Individual 

70.0 

68.4 

82.1 

67.9 

85.7 

81.0 

78.6 

64.3 

Small group (6 or less) 

65.0 

73.7 

85.7 

82.1 

85.7 

90.5 

64.3 

64.3 

Medium group (7 to 19) 

97.5 

97.4 

85.7 

82.1 

95.2 

100.0 

100.0 

92.9 

Large group (20 or more) 

47.5 

47.4 

50.0 

67.9 

52.4 

66.7 

50.0 

50.0 


Source: District program reports. Note. AC=academics-based program component; CC=character/citizenship 
component. 



Academic Character/citizenship 


□ Individual BSmall group DMedium group DLarge group 


Figure 3.2. Mean percent of districts using various grouping strategies to deliver 
academic and character/citizenship services (four cohorts). 


Academic Component 

> Instructional technology as well as more traditional instructional approaches were commonly 
used in the TASI academic components. 

Table 3.15 shows that districts relied on several technology-based programs for delivering aspects of 
TASI academic activities. Lightspan was the most commonly used program throughout the grant’s 
duration. General use of technology in academic activities (e.g., word processing, Internet) was also used 
by between 26% and 46% of districts. Overall, the use of specific technology programs declined, while 
general technology use increased slightly. 

The instructional approaches used by participating districts varied by year, but tutoring remained one of 
the most commonly used strategies throughout the four-year period. Districts also frequently reported that 
the academic component was enhanced through the provision of field trips and guest speakers. A few 
districts implemented commercial programs, such as Voyager or HOSTS (Helping One Student to 
Succeed). 
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Table 3.15. Activities Used in the Academics-Based Component (Percent) 


Activity 

Spr 2000 
N=40 

2000-01 

N=39 

2001-02 

N=32 

Fall 2002 
N=13 

Technology 

Lightspan 

40.0 

38.5 

25.0 

15.4 

Accelerated Math 

— 

12.5 

6.3 

— 

Accelerated Reader 

2.5 

6.3 

6.3 

— 

NovaNet/PLATO/A+LS 

5.0 

5.1 

— 

— 

General technology 

32.5 

25.6 

40.7 

46.2 

Technology lab 

2.5 

6.3 

— 

— 

Instructional approach 

Individualized 

— 

4.3 

9.4 

15.4 

Grouping 

25.0 

2.6 

12.5 

23.1 

Remediation 

— 

10.3 

15.7 

— 

Hands-on 

12.5 

28.2 

15.7 

38.5 

LEP/ESL 

2.5 

4.3 

6.3 

15.4 

Tutoring 

32.5 

53.8 

71.9 

69.2 

Projects 

10.0 

28.2 

50.0 

15.4 

TAAS/TAKS activities 

25.0 

28.2 

21.9 

30.8 

Curricular enhancement (e.g., field 
trips, speakers, etc.) 

32.5 

28.2 

18.8 

23.1 

Commercial academic program 

Voyager 

27.5 

12.8 

10.3 

15.4 

HOSTS 

10.0 

10.2 

- 

- 


Source: District program reports. Information unavailable in 2003. Note. Percent of districts 
citing an activity. A+LS=A+dvanced Learning System ®. 


> Time dedicated to technology declined somewhat over the grant period. 

In the first semester of the grant, 55% of districts dedicated 41% or more time to technology. During the 
remaining three grant years, only 38% to 43% of districts dedicated 41% or more of program time to 
technology use (Table 3.16). 


Table 3.16. Time Dedicated to Technology Use 
(Percent of Programs) 


Component 

Spring 

2000 

N=40 

2000-01 

N=2S 

2001-02 

N=21 

Fall 2002 
A=14 

20% or less 

27.5 

30.8 

25.0 


21-40% 

17.5 

30.8 

34.4 

14.3 

41-60% 

30.0 

15.4 

25.0 

28.6 

61-80% 


15.4 

9.4 

14.3 

8 1 % or more 


7.7 

6.3 

- 


Character Education/Citizenship Component 

> Districts used a combination of commercial programs ( Voyager, ROPES, Character Counts, 
etc.), external supports (guest speakers, field trips, community service, etc.), and other 
activities (athletics, fine arts, etc.) as part of the TASI character education/citizenship 
component. 

Voyager was the most consistently used commercial program in TASI character education/citizenship 
activities. Districts also joined with external entities to provide character/citizenship activities. These 
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linkages most often came in the form of guest speakers, community service activities, and field trips. 
Other activities students participated in frequently included athletics or fitness programs and 
arts/crafts/theater activities. Topics most frequently addressed in TASI programs include a citizenship, 
self-esteem, self-discipline, and social skills (Table 3.17). 


Table 3.17. Activities Used in the Character Education/Citizenship Component 


Activity 

Spring 2000 
N=40 

2000-01 

N=39 

2001-02 

N=32 

2002-03 

N=39 

Commercial programs 

Voyager 

22.5 

10.2 

39.6 

10.2 

ROPES 

17.5 

8.7 

12.5 

7.7 

Character Counts 

5.0 

11.3 

12.5 

5.1 

Boystown Social Skills 

5.0 

10.3 

— 

— 

Communities in Schools 

— 

6.3 

— 

5.1 

Teen Leadership 

— 

4.3 

6.3 

5.1 

7 Habits of Effective Teens 

7.5 

10.3 

— 

3.8 

Topics 

Citizenship 

35.0 

43.6 

43.8 

20.5 

Self-esteem 

22.5 

20.5 

9.4 

10.2 

Self-discipline 

30.0 

23.1 

12.5 

23.1 

Social skills (anger mgmt, 
teamwork) 

25.0 

41.0 

50.0 

30.8 

Life skills (career, drug 
awareness) 

10.0 

28.2 

18.8 

25.7 

MoraPethical issues 

2.5 

17.4 

31.3 

7.7 

External linkages 

Guest speakers 

37.5 

30.8 

31.3 

28.2 

Eield trips 

17.5 

25.6 

15.7 

10.2 

Community service 

20.0 

18.0 

18.8 

28.2 

Mentoring 

— 

13.0 

25.1 

15.4 

Physical development (karate, 
self-defense) 

15.0 

17.9 

6.3 

7.7 

Eacilitator (e.g.. Boys & Girls 
Club) 

7.5 

4.3 

18.8 

3.8 

Services/activities 

Athletics or fitness 

35.0 

23.1 

18.8 

38.5 

Arts/crafts/theater 

15.0 

10.3 

9.4 

17.9 

Student counseling 

7.5 

7.7 

9.4 

10.3 

Social services 

- 

8.7 

10.3 

- 

Clubs 

5.0 

5.1 

6.3 

7.7 

Projects (e.g., recycling, 
gardening) 

- 

12.8 

6.3 

12.8 


Parent and Mentor Component 

> Traditional means of communicating with parents (meetings, mail, telephone) and involving 
parents (training, volunteering) were most commonly used in TASI programs. 

Parent meetings or mail and telephone were the most frequently used methods of communication. Home 
visits or individual parent conferences seldom occurred. Sharing materials or progress reports with 
parents also occurred infrequently ( 4 % to 9% of districts). Districts provided other activities and services 
to parents, such as training or educational classes, volunteer opportunities, and occasions for parents to 
visit students’ classrooms or performances (Table 3.18). 
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Table 3.18. Activities Utilized in the Parent/Mentor Involvement Component 


Activity 

Spring 
2000 N=40 

2000-01 

N=39 

2001-02 

N=32 

Fall 2002 
N=13 

Communication 

Parent meetings 

40.0 

46.2 

53.2 

76.9 

Communication (e.g., mail, 
phone) 

12.5 

25.6 

37.5 

38.5 

Language accommodation 

7.5 

12.8 

- 

15.4 

Home visits 

- 

12.8 

- 

- 

Parent conferences 

- 

5.1 

6.3 

7.7 

Parent center 

- 

5.1 

- 

- 

Information 

Materials 

- 

4.3 

9.4 

- 

Progress reports 

- 

8.7 

6.3 

- 

Activities 

Parent training/education 

50.0 

43.6 

37.6 

38.5 

Volunteer opportunities 

30.0 

35.9 

50.1 

53.8 

Observations (e.g., classes, 
performances) 

32.5 

12.8 

15.7 

7.7 

Projects involving parents 

7.5 

17.9 

12.5 

15.4 

Parent survey 

5.0 

4.3 

6.3 

7.7 

Parent counseling 

2.5 

- 

6.3 

- 


> Parent involvement most commonly meant attending a parent night or serving as a helper. In 
contrast, mentors most often served as tutors or speakers. 

In describing the nature of parent involvement, TASI programs most frequently reported that parents 
attended parent nights (between 71% and 90% of programs) or served as helpers (between 68% and 86% 
of programs). Mentors more often served as tutors, speakers, or helpers (Table 3.19). 


Table 3.19. Parent and Mentor Activities (Percent of Programs) 


Category 

Spr 2000 
A=40 

2000-01 

N=2S 

2001-02 

A=21 

Fall 2002 
A=14 

Parent involvement 

Parent Night attendees 

89.7 

85.7 

85.7 

71.4 

Helpers 

71.8 

67.9 

85.7 

78.6 

Instructional aides 

38.5 

42.9 

52.4 

50.0 

Speakers 

33.3 

42.9 

57.1 

35.7 

Tutors 

30.8 

46.4 

52.4 

50.0 

Teachers 

20.5 

25.0 

47.6 

35.7 

Mentor involvement 

Tutors 

64.1 

64.3 

61.9 

78.6 

Speakers 

61.5 

50.0 

61.9 

57.1 

Helpers 

56.4 

46.4 

52.4 

64.3 

Teachers 

53.8 

50.0 

52.4 

50.0 

Instructional aides 

48.7 

42.9 

52.4 

57.1 

Parent Night attendees 

48.7 

42.9 

47.6 

42.9 


Source: District program reports. Information unavailable in 2003. 
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Participation in Additionai Activities 

> TASI students most commonly engaged in extracurricular and community service activities. 

Districts also reported on the participation of TASI students in other school activities. Table 4.20 shows 
that the percentage of students participating in activities varied hy term. However, TASI students were 
most likely to he involved in extracurricular activities and least likely to he involved in intramurals. 
Differences hy eligibility emerged — students with no risk factors had greater participation rates in 
extracurricular activities and student clubs, while at-risk students tended to be more involved in 
community service activities (Table 3.20). 


Table 3.20. Student Participation in Additional School Activities (Percent) 


School Activity 

Spring 

2000 

2000-01 

2001-02 

2002-03 

All TASI participants 

Community service 

17.2 

11.6 

17.2 

19.6 

Extracurricular activities 

26.5 

17.5 

21.6 


Intramural activities 

15.0 

3.4 

7.5 

8^ 

Student clubs 

12.6 

10.9 

14.1 


Meets state Compensatory Education requirements 

Community service 

16.3 

15.7 

234.6 

21-8 

Extracurricular activities 

25.2 

20.1 

21.1 


Intramural activities 

14.4 

4.0 

9.5 

8.4 

Student clubs 

12.3 

11.7 

16.5 

14.8 

Meets other risk factors 

Community service 

18.5 

12.9 

12.3 

20.6 

Extracurricular activities 

32.3 

19.4 

21.8 


Intramural activities 

14.3 

3.1 

6.5 


Student clubs 

11.7 

11.5 

11.7 

16A 1 

No at-risk characteristics 

Community service 

23.0 

11.9 

18.4 

16.7 

Extracurricular activities 

39.1 

22.3 

34.2 

32.1 

Intramural activities 

16.4 

3.9 

8.9 

7.6 

Student clubs 

18.0 

16.0 

20.6 



STUDENT OUTCOMES 
Passing Core-Content Courses 

> The majority of TASI students (84% to 89%) passed core content-area courses. 

Between 84% and 89% of TASI students passed core-content courses, although passing rates were 
slightly lower for mathematics courses. Students with no risk characteristics had the highest passing rates 
(more than 90% for all core-content areas each semester). Passing rates for participants meeting 
Compensatory Education requirements or having other risk factors generally lagged approximately 10 
percentage points below their non-at-risk counterparts (Table 3.21). 
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Table 3.21. Percentage of Students Passing Core-Content Area Courses 


Time Period 

Language Arts 

Mathematics 

Social Studies 

Science 

N 

Pass 

N 

Pass 

N 

Pass 

N 

Pass 

All TASI participants 

Spring 2000 

7,532 

83.9 

7,568 

83.8 

6,598 

87.6 

6,425 

86.7 

2000-01 

13,882 

87.4 

14,133 

83.8 

14,005 

88.5 

14,011 

87.1 

2001-02 

19,100 

86.7 

19,302 




18,925 

88.1 

2002-03 

24,922 

88.4 

25,074 

85.8 


89.2 

25,005 

88.8 

Meets state Compensatory Education requirements 

Spring 2000 

4,812 

85.4 

4,823 

81.8 

4,797 

86.2 

4,715 

84.8 

2000-01 

5,206 

84.4 

5,396 

80.7 

5,350 

85.9 

5,352 

84.7 

2001-02 

8,656 

86.1 

8,746 

82.9 

8,712 

86.2 

8,591 

87.5 

2002-03 

11,416 

85.4 

11,527 

81.5 

11,547 

86.1 

11,542 

85.6 

Meets other risk factors 

Spring 2000 

933 

85.2 

938 

79.0 

931 

85.9 

930 


2000-01 

3,663 

85.0 

3,695 

80.9 

3,662 

86.8 

3,664 



3,982 

86.2 


83.7 

4,021 

88.3 

3,954 



6,397 

87.5 


86.0 

6,416 

89.2 

6,401 


No at-risk characteristics 

Spring 2000 

722 

90.9 

749 

91.5 

745 

91.9 

739 

92.4 

2000-01 

2,412 

95.2 

2,424 

92.5 

2,409 

95.7 

2,407 

94.6 

2001-02 

3,421 

95.9 

3,423 

94.4 

3,426 

96.0 

3,385 

96.3 

2002-03 

6,272 

95.0 

6,279 

93.5 

6,289 

95.4 

6,280 

95.7 


Note. Passing percentage based on number of courses taken. Columns may not sum because 
some students lack eligibility data. 


Discipline Referrals 

> Discipline problems are an issue for some TASI students. Nearly one-fifth of students had four 
or more office referrals (17% to 20%), and a small percentage of students were referred to 
alternative education programs (about 5%) or juvenile justice programs (less than 1%). 

Across the four years, approximately 20% of TASI students meeting Compensatory Education 
requirements or other risk factors have been referred to the office four or more times. In comparison, 
approximately 10% of students with no risk characteristics have received four or more office referrals. 
Placements in alternative education or juvenile justice programs remained low across semesters, however 
alternative education placements increased somewhat over time. Not surprisingly, students with no risk 
characteristics had fewer alternative education or juvenile justice placements (Table 3.22). 
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Table 3.22. Student Disciplinary Referrals (Percent) 


Referral Category 

Spring 

2000 

2000-01 

2001-02 

2002-03 

All TASI participants 

No office referrals 

— 

52.7 

54.2 

55.1 

1 to 3 office referrals 

— 

28.8 

26.0 

27.6 

4 or more office referrals 

1.5 

18.6 

19.8 

17.3 

AEP placement 

0.2 

3.3 

5.2 

5.6 

JJAEP referral 

0.5 

0.6 

0.4 

0.6 

Meets state Compensatory Education requirements 

No office referrals 

— 

48.0 

51.1 

50.0 

1 to 3 office referrals 

— 

30.6 

27.9 

29.2 

4 or more office referrals 

1.4 

21.4 

21.0 

20.8 

AEP placement 

0.1 

3.8 

5.5 

7.4 

JJAEP referral 

0.4 

0.9 

0.6 

0.8 

Meets other risk factors 

No office referrals 

— 

51.6 

50.4 

53.8 

1 to 3 office referrals 

— 

29.5 

25.9 

28.8 

4 or more office referrals 

2.8 

19.0 

23.7 

17.4 

AEP placement 

0.0 

4.5 

7.2 

4.6 

JJAEP referral 

2.2 

0.7 

0.4 

0.5 

No at-risk characteristics 

No office referrals 

— 

67.6 

68.9 

67.6 

1 to 3 office referrals 

— 

22.1 

23.0 

23.3 

4 or more office referrals 

2.2 

10.3 

8.1 

9.1 

AEP placement 

0.0 

2.2 

1.7 

3.3 

JJAEP referral 

0.3 

0.4 

0.3 

0.3 


Note. Office referral data not collected in spring 2000 (denoted by — ). 

AEP = Alternative Education Program. JJAEP = Juvenile Justice Alternative 
Education Program. 


Comparisons Between TASI and Non-TASI Students 

Estimating the impact of a program intervention requires comparisons between program participants and 
a comparable group of non- served students. Although we have data for all grades 6-8 students in each 
participating TASI campus, it is impossible to form a true TASI comparison group. By program mandate, 
only students at risk of academic failure and/or at risk of committing juvenile offenses received TASI 
services. Thus, although non-served students might include some students at risk, the majority of students 
in the comparison group will have fewer risk factors. In an attempt to create a more comparable group of 
non-served students, we have restricted the comparison group to those students who are similar to the 
TASI-served students on seven dimensions: grade level, ethnicity, gender, economic disadvantage status 
(based on eligibility for free or reduced-price lunch), limited English proficiency, whether or not they 
repeated their current grade, and district attended. The characteristics of TASI and selected students are 
given in Table 3.23. 
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Table 3.23. Characteristics of Students Included in Comparison Groups 


Characteristic 

Cohort 1 
N=15,206 

Cohort 2 
N=23,974 

Cohort 3 
N=33,814 

Cohort 4 
N=38,802 

Grade 


Sixth 

33.8 

31.4 

33.5 

30.3 

Seventh 

37.3 

37.2 

34.7 

36.9 

Eighth 

29.0 

31.4 

31.8 

32.8 

Gender 


Male 

52.0 

52.3 

51.1 

50.0 

Female 

48.0 

47.7 

48.9 

50.0 

Race/Ethnicity 


Hispanic 

55.4 

52.5 

53.9 

55.9 

African American 

23.1 

21.0 

21.3 

22.4 

White 

20.1 

24.5 

22.5 

19.8 

Other/NA 

1.4 

2.0 

2.3 

1.8 

Repeating grade level 

1.3 

1.4 

1.4 

1.4 

Limited English proficient 

15.8 

11.7 

12.6 

13.6 

Economically disadvantaged 

63.8 

63.8 

66.4 

68.7 


Of the 15,206 students in cohort 1, half (7,603) are TASl students and half are non-TASl students. 

Cohorts 2 through 4 have a greater number of students because the number of students served in TASl 
programs increased during those years. Again, half of the students are in TASl and half are non-TASl. 
Most demographic characteristics are similar across all four cohorts. Changes across time reflect changes 
in the demographics of TASl served students. Over time, the percentage of male students. White 
students, and sixth-grade students decreased, whereas the percentage of economically disadvantaged 
students increased. 

Attendance Rates 

Using data from PEIMS, the effect of the TASl program on student attendance rates is explored using 
time-series analysis. As stated previously, each program year forms one cohort of students, and each 
cohort of students is analyzed separately. Five years of attendance data are available for cohort 1 students 
(1997 through 2002), five years for cohort 2 (1998 to 2003), four years for cohort 3 (1999 to 2003), and 
three years for cohort 4 (2000 to 2003). The analysis is restricted to matched students with valid data for 
all years. The number of students included in the analysis is given in Table 3.24. Of the 14,334 students in 
cohort 1, 14,155 are in their grade (6th, 7th, or 8th) for the first time and 179 are repeating their grade 
level. There are 22,130 students in cohort 2; 32,621 in cohort 3; and 38,802 in cohort 4. The vast majority 
of students are in their grade for the first time. 
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Table 3.24. Number of Students Included In Attendance Rate Analysis 


Cohort/Student Category 

TASI 

Non-TASI 

Total 

Cohort 1 
1999-00 

All students 

7,201 

7,133 

14,334 

First time in grade 

7,115 

7,040 

14,155 

Repeating grade 

86 

93 

179 

Cohort 2 
2000-01 

All students 

11,160 

10,970 

22,130 

First time in grade 

11,022 

10,829 

21,851 

Repeating grade 

138 

141 

279 

Cohort 3 
2001-02 

All students 

16,348 

16,273 

32,621 

First time in grade 

16,127 

16,052 

32,179 

Repeating grade 

221 

221 

442 

Cohort 4 
2002-03 

All students 

19,401 

19,401 

38,802 

First time in grade 

19,126 

19,126 

38,252 

Repeating grade 

275 

275 

550 


Note. “First time in grade” refers to students who are enrolled in 6th, 7th, or 8th grade 
for the first time. “Repeating grade” refers to students who remained at the same 
grade level as the previous school year 


> In general, TASI students’ attendance rates have not improved over time. Attendance rates for 
TASI students, however, are consistently higher than rates for non-TASI students. 

Longitudinal attendance rate data for students in each of the four cohorts show that attendance rates are 
near state averages (95%) hut decline as students progress to higher grade levels, except for cohort 4 
students who increased their attendance rate over time. Although TASI students have better attendance 
rates than non-TASI students, differences cannot he attrihuted to the after-school program since the 
difference between TASI and non-TASI students remains relatively stable across years. These data, 
however, do show that TASI students have a more positive school attendance profile than 
demographically similar students who did not participate in TASI. Attendance rate differences may reflect 
motivational or altitudinal characteristics of students that make them more strongly connected to the 
school or more willing to participate in educational activities that extend the school day. 


Table 3.25. Longitudinal Attendance Rates for All Students 



Cohort I 
1999-00 

Cohort 2 
2000-01 

Cohort 3 
2001-02 

Cohort 4 
2002-03 

Year 

TASI 

Students 

Non-TASI 

Students 

TASI 

Students 

Non-TASI 

Students 

TASI 

Students 

Non-TASI 

Students 

TASI 

Students 

Non-TASI 

Students 

1997-1998 

96.7 

96.4 







1998-1999 

96.1 

95.8 

96.6 

96.3 





1999-2000 

95.8 

95.1 

96.6 

96.0 

96.8 

96.4 



2000-2001 

94.7 

94.1 

95.7 

94.9 

96.3 

95.6 

93.1 

92.0 

2001-2002 

93.3 

92.8 

95.0 

94.2 

95.7 

94.8 

96.2 

95.5 

2002-2003 



93.7 

93.1 

94.8 

94.0 

95.7 

94.7 

Change 

-3.4 

-3.6 

-2.9 

-3.2 

-2.0 

-2.4 

h-2.6 

+ 1.1 


Student attendance rates, disaggregated by students in a grade for the first time and those who are 
repeating a grade level, are displayed in Figures 3.3 through 3.6 for the four student cohorts. 
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> First-time students have higher attendance rates than students repeating a grade ievei for 
both TASi and non-TASi comparison groups. Aithough attendance rates for repeating TASi 
students deciined over time, a positive change was observed during the TASi program 
impiementation year for the iast three cohorts. 

Disaggregation of cohort 1 students’ attendance data in Figure 3.3 by first- time and repeating students 
shows that first-time students have notably higher attendance rates. In addition, TASI students had 
slightly higher attendance rates than non-TASI students. For all comparison groups, attendance rates 
decline over time. The attendance rate decline for students repeating a grade stabilized slightly between 
the 1998-99 and the 1999-00 school years, but the plateau was similar for both TASI and non-TASI 
students. 



Figure 3.3. Attendance rates for cohort 1, by students in their grade for the 
first time and students repeating a grade ievei. 


Figure 3.4 shows that cohort 2 students also exhibited a downward trend in attendance rates across 
years — ^however, the trend is altered for TASI participants who were repeating their grade level. This 
group of TASI students actually experienced an increase in attendance rates for the program year (2000- 
01). Unfortunately, this positive trend was not maintained, as the general attendance rate decline resumed 
for repeating students the following school year (2001-02). 
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Figure 3.4. Attendance rates for cohort 2, by students in their grade for the 
first time and students repeating a grade ievei. 


Results for attendance rates are similar for cohort 3 students. TASI participants who were repeating a 
grade experienced a slight attendance increase for the TASI program year (2001-02) with a resumption of 
a downward trend the following year. 



Figure 3.5. Attendance rates for cohort 3, by students in their grade for the 
first time and students repeating a grade ievei. 
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Attendance results for cohort 4 students differ from the other cohorts. For cohort 4 students in their grade 
for the first time, attendance rates spiked the year before TASI participation then declined slightly during 
the TASI year. For students repeating a grade level, TASI students had markedly higher attendance rates 
that were sustained during the TASI program year. 



Figure 3.6. Attendance rates for cohort 4, by students in their grade for the 
first time and students repeating a grade ievei. 


State-Level Assessments 

In an effort to evaluate the effect of TASI participation on student academic performance, TAAS reading 
and mathematics test scores were merged with the student-level database. TAAS performance is 
measured by whether or not a student passed the reading and mathematics tests. Available scores include 
those for the year prior to TASI participation and at the end of the TASI program year. The sample is 
restricted to students who have TAAS data for both testing years. ^ Analyses were conducted separately 
for first-time students and students repeating their grade level.^ During the fourth cohort year (2002-03), 
the TAAS test was replaced with the TAKS and therefore no comparison can be made to a pretest. The 
TAKS posttest is given for cohort 4 in tables 3.26 and 3.27. 

Students in Grade Level for the First Time 

> TASI students had lower TAAS passing rates for both reading and mathematics compared to 
non-TASI students, but the achievement gap between groups narrowed slightly for cohorts 1 
to 3. Despite apparent progress, the achievement gap between TASI and non-TASI students 
Increased In both subject areas for cohort 4 students who completed the TAKS assessments. 

Table 3.26 shows that only 67.4% of TASI students in cohort 1 passed TAAS reading before participating 
in an after-school program, whereas 79.5% of non-TASI students passed (an achievement gap of 11.7 
percentage points). At the end of the TASI program year, 72.5% of TASI students passed TAAS reading 
compared to 82% of non-TASI students (for an achievement gap of 9.5 percentage points). Although 


* Absent, ARD exempt, and LEP exempt student scores are not used. 
^ Results for all students parallel those for first-time students. 
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non-TASI students outperformed TASI students on both assessments, the performance gap narrowed 
slightly (2.2 percentage points). This slight narrowing of the achievement gap across cohorts was 
consistent for both TAAS reading and mathematics tests, with the largest gap reduction occurring for 
TAAS reading in cohort 2 (3.2 percentage points) and the smallest for TAAS mathematics in cohort 3 
(1.2 points). 

Despite encouraging TAAS results, outcomes for cohort 4 students as measured by the new state 
assessment — Texas Assessment of Knowledge and Skills (TAKS) — show that the achievement gap 
between comparison groups widened, especially in math. 


Table 3.26. TAAS/TAKS Reading and Mathematics Passing Rates, First Time Students 


Cohort/ 

Test Category 

Reading 

Mathematics 

TASI 

Students 

Non-TASI 

Students 

Achievement 

Gap 

TASI 

Students 

Non-TASI 

Students 

Achievement 

Gap 

Cohort 1 
1999-00 
N= 11,692 

Pretest 

67.4% 

79.2% 

11.7 

71.0% 

82.2% 

11.1 

Posttest 

72.5% 

82.0% 

9.5 

75.7% 

84.2% 

8.4 

Cohort 2 

2000-01 

N=18,588 

Pretest 

73.9% 

82.5% 

8.7 

80.3% 

86.8% 

6.4 

Posttest 

80.2% 

85.7% 

5.5 

83.5% 

87.5% 

4.0 

Cohort 3 
2001-02 
N=26,400 

Pretest 

80.2% 

85.5% 

5.3 

86.9% 

90.4% 

3.5 

Posttest 

85.5% 

88.8% 

3.3 

88.1% 

90.4% 

2.3 

Cohort 4 
2002-03 
N=3 1,860 

Pretest" 

- 

- 

- 

- 

- 

- 

Posttest 

78.5% 

83.0% 

4.5 

60.9% 

67.9% 

7.0 


Note. Cohorts have equal numbers of TASI and non-TASI students (cohort 1-5,846 students; cohort 2-9,294 students; 
cohort 3-13,200 students; and cohort 4 - 15,930). 

■'Texas first administered the TAKS assessment in 2002-03; thus, pretests are unavailable. 


Students Repeating a Grade Level 

> For the small number of students repeating their grade level, the TAAS passing rate gap was 
narrowed between TASI and non-TASI student cohorts. In fact, TASI students in cohort 3 
passed TAAS reading at a higher rate than non-TASI students. Moreover, TASI students 
maintained their passing rate advantage over non-TASI students on TAKS reading and nearly 
equaled comparison-group students for mathematics. 

Comparisons in Table 3.27 show that a small number of students repeating their grade level had far lower 
TAAS passing rates than students in their grade level for the first time. Cohort 1 TASI students had a 
passing rate of only 24.6% for TAAS reading, whereas non-TASI students had a passing rate of 43.9% (a 
gap of 19.3 percentage points). Both student groups substantially increased their passing rates by the end 
of the program year, with TASI students having a slightly larger increase (5.3 percentage points). Given 
the extremely low starting point, the change may not reflect program effects but may instead result from 
the fact that students with extremely low scores have no place to go other than up (known as regression to 
the mean). Examining TAAS mathematics passing rates for cohort 1 students, TASI and non-TASI 
students have similar starting points (28.1% and 33.3%, respectively), and because both groups have low 
pretest scores, regression to the mean affects both groups equally. Consequently, the 1.8 percentage point 
gap reduction for TAAS mathematics is considerably less than for TAAS reading. 
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Table 3.27. TAAS/TAKS Reading and Mathematics Passing Rates, 
Students Repeating Their Grade 


Cohort/ 

Test Category 

Reading 

Mathematics 

TASI 

Students 

Non-TASI 

Students 

Achievement 

Gap 

TASI 

Students 

Non-TASI 

Students 

Achievement 

Gap 

Cohort 1 
1999-00 
N=114 

Pretest 

24.6% 

43.9% 

19.3 

28.1% 

33.3% 

5.3 

Posttest 

52.6% 

66.7% 

14.0 

52.6% 

56.1% 

3.5 

Cohort 2 

2000-01 

N=188 

Pretest 

40.4% 

47.9% 

7.4 

44.7% 

54.3% 

9.6 

Posttest 

63.8% 

70.2% 

6.4 

71.3% 

72.3% 

1.1 

Cohort 3 
2001-02 
N=262 

Pretest 

49.6% 

51.9% 

2.3 

46.6% 

57.3% 

10.7 

Posttest 

78.6% 

66.4% 

h-12.2 

75.6% 

77.1% 

1.5 

Cohort 4 
2002-03 
N=350 

Pretest"" 

- 

- 

- 

- 

- 

- 

Posttest 

61.1% 

57.7% 

h-3.4 

36.6% 

40.0% 

3.4 


Note. Cohorts have equal numbers of TASI and non-TASI students (cohort 1-57 students; cohort 2-94 students; 
cohort 3-131 students; cohort 4 - 175 students). 

■"Texas first administered the TAKS assessment in 2002-03; thus, pretests are unavailable. 


The most dramatic change in TAAS passing rates occurred for cohort 3 students in the area of reading. 
Comparison groups for cohort 3 had similar passing rates for the year prior to participation in TASI: 
49.6% for TASI and 51.9% for non-TASI students, an achievement gap of only 2.3 percentage points. By 
the end of the program year, TASI students passed TAAS reading at a rate of 78.6%, while only 66.4% of 
non-TASI students passed. TASI students not only closed the achievement gap hy 14.5 percentage 
points, hut more importantly, they outperformed non-TASI students. 

For Cohort 4, TASI students repeating a grade level maintained their positive advantage over non-TASI 
students on the TAKS reading assessment. The 61.1% of TASI students passing TAKS reading exceeded 
comparison students by 3.4 percentage points. However, for both TASI and non-TASI students passing 
rates for TAKS mathematics dropped dramatically compared to TAAS (only 36.6% and 40.0% passed, 
respectively). The achievement gap between TASI and non-TASI students also increased. 

The TAAS/TAKS reading passing rates at the end of each cohort year are displayed graphically in 
Figure 3.7 for students in their grade for the first time and students repeating their grade level. This 
additional representation of the data presented previously in Tables 3.26 and 3.27 illustrates how the gap 
between TASI and non-TASI students’ TAAS passing rates for reading closed over time. First-time 
students experienced higher passing rates than repeating students. For students repeating a grade, TASI 
students in cohort 3 had a significantly higher passing rate while the rate for non-TASI students remained 
relatively stable. 
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Figure 3.7. Posttest passing rates for TAAS reading (cohorts 1-3) and TAKS reading 
(cohort 4), by students in their grade for the first time and students repeating a 
grade ievei. 
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Figure 3.8. Posttest passing rates for TAAS mathematics (cohorts 1-3) and TAKS 
mathematics (cohort 4), by students in their grade for the first time and students 
repeating a grade ievei. 
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Grades 6, 7, and 8 

> Grade-level results show that TASI students generally had lower TAAS passing rates for both 
reading and mathematics than non-TASI comparison group students across all grade levels. 
Consistent with results for all students, the achievement gap between TASI and non-TASI 
students narrowed slightly. 

TAAS reading and mathematics passing rates for grades 6, 7, and 8 are included in Tables C.2, C.3, and 
C.4 in Appendix C. Overall, grade-level results mirror trends for all students. Non-TASI students have 
higher TAAS passing rates for all grade levels. TASI students in cohort 1 had low passing rates ranging 
from 68.5% (6th grade) to 77.4% (8th grade). TAAS passing rates improved for TASI students in cohorts 
2 and 3 across all grade levels, although rates remained below those for non-TASI students. Eighth-grade 
TASI students in cohort 3, however, passed TAAS reading at nearly identical rates as non-TASI students 
(91.3% versus 91.9%). TAKS results for cohort 4 students are less promising than previous cohorts 
TAAS scores, especially for mathematics. Not only were the actual passing rates lower, but across the 
three grade levels the achievement gap between TASI and non-TASI students increased, especially for 
TAKS math. For non-TASI students in 6th grade, 72.1% passed TAKS math compared to 62.1% of TASI 
students. Thus, there was also a 7 percentage point gap in sixth graders TAKS reading favoring non-TASI 
students. The gap was progressively narrowed for 7th and 8th graders, but the gaps were always greater 
than differences between the previous cohort’s TAAS scores. 

Retention Rates 

Another important measure of academic success is student promotion to the next grade level. For this 
analysis, student retention data are only available for cohorts 1 through 3. First, we examine retention 
rates for all students in the three student cohorts. Next, retention rates are explored for students who are 
enrolled in 6th, 7th, or 8th grade for the first time and students who are repeating the same grade as the 
previous year. 

> For TASI students, retention rates declined across cohorts, and In cohort 3, TASI students had 
lower retention rates than non-TASI students. 

> For the small number of students repeating their grade level, across-cohort trends show that 
TASI students had lower retention rates than non-TASI students over time. 

Retention in grade level does not appear to be a large problem for middle school students. As Table 3.28 
shows, the retention rates for each student cohort (both TASI and non-TASI) are small, with the rate for 
all students 3% or less. This is not true, however, for the few students repeating their grade level. These 
students have retention rates ranging from 3% to 12%. As a result of the low numbers of repeating 
students, results for all students and first-time students are nearly identical. 
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Table 3.28. Retention Rates for All Students, by Cohort and Comparison group 



Students repeating their grade level had much higher retention rates than first-time students, for both 
TASI and non-TASI comparison groups. For cohort 1, TASI students repeating a grade had a retention 
rate twice the rate of non-TASI students (6% versus 3%). Over time, retention rates for TASI students 
repeating a grade level declined, from 6% (cohort 1) to 3% (cohort 3). In contrast, retention rates for non- 
TASI students increased. By cohort 3, the retention rate for non-TASI repeating students was 6.5%, more 
than twice the rate for TASI students. 

Figure 3.9 illustrates the retention rate differences for TASI and non-TASI comparison groups across 
cohorts. For TASI students in cohort I, 3% were retained in their grade, a higher proportion than the 2.5% 
rate for non-TASI students. TASI and non-TASI students in cohort 2 had similar retention rates (2.9% 
and 3% respectively). For cohort 3, retention rates for TASI students (2.2%) were slightly lower than for 
non-TASI students (2.5%). 



Figure 3.9. Retention rates for all students, by cohort and comparison group. 

> Although student retention varies by grade level, TASI students’ retention rates declined 
across all cohorts for all grade levels. By cohort 3, seventh- and -eighth grade TASI students 
had lower retention rates than non-TASI students. 
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For all student cohorts, grade-level retention rates ranged from 1.8% to 4.1% (as shown in Table 3.29). 
Seventh graders consistently had the highest retention rates for both TASI and non-TASI students. 
Retention rates for non-TASI students remained relatively stable across cohorts, with a variation of no 
more than 0.7 percentage points from cohort 1 to cohort 3 for any grade level. Sixth-grade TASI students 
had a steady decline in retention rates over time, although in cohort 3, rates remained slightly higher than 
those for non-TASI students. Seventh-grade TASI students in cohort 1 had the highest retention rate of 
any group (4.1%) and experienced the largest decline, such that by cohort 3, they had a lower retention 
rate than non-TASI students. Eighth-grade TASI students also experienced a decline in retention rates, 
and by cohort 3, they too had a rate lower than non-TASI students. 


Table 3.29. Retention Rates for All Students, by Grade Level 


Grade/Cohort 

TASI Students 

Non-TASI Students 

Number of 
Students 

Percent 

Retained 

Number of 
Students 

Percent 

Retained 

Sixth 

Cohort 1 (1999-00) 

2,567 

2.6 

2,567 

1.9 

Cohort 2 (2000-01) 

3,759 

2.8 

3,759 

2.3 

Cohort 3 (2001-02) 

5,670 

2.2 

5,670 

1.9 

Seventh 

Cohort 1 (1999-00) 

2,834 

4.1 

2,834 

3.5 

Cohort 2 (2000-01) 

4,461 

3.5 

4,461 

3.7 

Cohort 3 (2001-02) 

5,858 

2.4 

5,858 

3.2 

Eighth 

Cohort 1 (1999-00) 

2,202 

2.2 

2,202 

2.1 

Cohort 2 (2000-01) 

3,764 

2.4 

3,764 

2.8 

Cohort 3 (2001-02) 

5,377 

1.8 

5,377 

2.3 


Association of Student- and District-Levei Factors 
with Academic Achievement 

The relationships between TASI student and district characteristics and TAAS reading and mathematics 
TLI scores were explored using hierarchical linear modeling (HEM). HEM was the preferred analytical 
method because, in most cases, students within school districts are similar to each other because of 
selection processes and common backgrounds. Consequently, measures within school districts are not 
independent. Rather, the correlation between measures of students from the same district will tend to be 
higher than the correlation between measures of students from different districts. Not only does 
hierarchical linear modeling make no assumption about independence, it estimates the degree of 
dependence of measures and uses this estimate in the calculation of the precision with which treatment 
effects are estimated. 

Separate HEM analyses were conducted using TASI participants in 2000-01 (cohort 2) and 2001-02 
(cohort 3). Cohort 2 included 7,601 students in 42 school districts and cohort 3 included 11,172 students 
in 45 districts. These students participated in extended-day programs in their respective TASI year, 
constituted at least 10 students in the school district, and had TAAS scores from the year prior to TASI 
participation (2000 for cohort 2 and 2001 for cohort 3) and the end of the TASI participation year (2001 
for cohort 2 and 2002 for cohort 3). These students were used to investigate the effect of TASI student 
and district characteristics on TAAS reading and mathematics TEI scores. 

The specific student- and district-level variables along with their descriptive statistics are reported in 
Table 3.30 for TAAS reading and mathematics. The student-level variables included gender (1 if male, 0 
if female), economic status (1 if disadvantaged, 0 if not), prior retention (1 if retained the year prior to 
TASI, 0 if not), school attendance (average rate for TASI participation year), TAAS reading and 
mathematics TEI pretest scores (2000 for cohort 2 and 2001 for cohort 3), days participated in TASI, 
minority status (1 minority, 0 if White), and grade level in the TASI participation year. 
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District-level variables included the district TASI mean TAAS pretest score (achievement contextual 
effect), the district proportion of disadvantaged TASI students (economically disadvantaged contextual 
effect), and the district per-pupil TASI expenditure for that year. 


Table 3.30. Descriptive Statistics for TAAS Reading and Mathematics Data 


Variable Name 

N 

Mean 

SD 

Minimum 

Maximum 

Cohort 2 (2000-01) Student-Level Descriptive Statistics 

Grade 6 

7,601 

.31 

.46 

0 

1 

Grade 7 

7,601 

.37 

.48 

0 

1 

Gender 

7,601 

.50 

.50 

0 

1 

Minority 

7,601 

.74 

.44 

0 

1 

Economically disadvantaged 

7,601 

.64 

.48 

0 

1 

Prior retention 

7,601 

.02 

.13 

0 

1 

School attendance 

7,601 

96.14 

4.09 

50.3 

100.0 

TAAS reading 2000 

7,601 

78.55 

14.52 

29.0 

101.0 

TAAS reading 2001 

7,601 

81.06 

13.80 

18.0 

100.0 

TAAS mathematics 2000 

7,601 

78.64 

11.36 

12.0 

93.0 

TAAS mathematics 2001 

7,601 

79.43 

9.77 

13.0 

93.0 

Days participated in TASI 

7,601 

28.99 

31.47 

0 

174.0 

Cohort 2 (2000-01) District-Level Descriptive Statistics 

Economically disadvantaged 

42 

63.70 

17.86 

31.08 

97.62 

Reading achievement (2001) 

42 

76.70 

4.75 

64.42 

84.53 

Mathematics achievement (2001) 

42 

77.21 

3.17 

66.86 

82.07 

TASI expenditure per pupil 

42 

727.52 

652.54 

42.87 

3,580.68 

Days TASI offered 

42 

54.58 

25.43 

3.40 

112.25 

Cohort 3 (2001-02) Student-Level Descriptive Statistics 

Grade 6 

11,172 

0.33 

0.47 

0 

1 

Grade 7 

11,172 

0.35 

0.48 

0 

1 

Gender 

11,172 

0.51 

0.50 

0 

1 

Minority 

11,172 

0.75 

0.43 

0 

1 

Economically disadvantaged 

11,172 

0.63 

0.48 

0 

1 

Prior retention 

11,172 

0.01 

0.12 

0 

1 

School attendance 

11,172 

96.16 

3.97 

41.38 

100.0 

TAAS reading 2001 

11,172 

81.54 

13.97 

17.0 

101.0 

TAAS reading 2002 

11,172 

84.10 

13.44 

11.0 

100.0 

TAAS mathematics 2001 

11,172 

80.92 

9.60 

34.0 

93.0 

TAAS mathematics 2002 

11,172 

81.34 

9.36 

13.0 

93.0 

Days participated in TASI 

11,172 

24.75 

26.43 

0 

189.0 

Cohort 3 (2001-02) District-Level Descriptive Statistics 

Economically disadvantaged 

45 

62.08 

20.12 

18.18 

99.61 

Reading achievement (2001) 

45 

79.84 

4.85 

67.25 

88.59 

Mathematics achievement (2001) 

45 

80.16 

3.09 

70.79 

85.65 

TASI expenditure per pupil 

45 

397.18 

403.38 

40.55 

2,445.65 

Days TASI offered 

45 

50.52 

26.58 

4.7 

102.04 
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Student-Level Variables 

> After controlling for the effect of student-level characteristics (academic and social 
background), there was no positive relationship between days students spend in TASI (up to 
189 days) and TAAS scores. 

> A higher school attendance rate for a student was associated with higher TAAS reading and 
mathematics scores. 

> Students in grade 8 had the highest adjusted TAAS scores, while those in grade 6 had the 
lowest. 

Table 3.31 shows that student-level predictors associated with higher TAAS reading scores include being 
female, economically advantaged, non-minority, repeating the current grade level, school attendance, and 
attending grade 8. Higher TAAS reading pretest scores were also strongly associated with higher posttest 
reading scores. For example, for cohort 2 TASI students, a unit increase in 2000 TAAS reading scores 
was associated with a 0.64 unit increase in 2001 TAAS reading scores. School attendance was also a 
significant predictor of TAAS reading and mathematics TLI scores. By way of example, in cohort 2, a 1% 
increase in the attendance rate resulted in a 0.22 TLI unit increase in mathematics posttest scores, net of 
other level 1 predictors. However, there was no significant relationship between TASI instructional days 
and TAAS reading TLI scores. This is consistent with other studies of after-school programs showing 
limited impact on student achievement (e.g., Dynarski et al., 2003). 

Similar student-level predictors were associated with higher TAAS mathematics scores. These included 
being non-minority, attending grade 8, having a higher TAAS mathematics pretest score, and school 
attendance. As with the reading analyses, there was no relationship between the number of TASI 
instructional days and subsequent TAAS mathematics scores. For cohort 3, higher TAAS reading and 
mathematics scores were not associated with being female or economically advantaged. 

District-Level Variables 

> After controlling for the effect of student characteristics (academic and social background) 
and district social and academic contexts, there was no significant relationship between TASI 
dollars per pupil and TAAS reading and mathematics scores. 

Contextual effects occur when the aggregate of a student-level characteristic is related to an outcome 
variable, even after controlling for the effect of the student-level characteristic. An example would be 
average social class of a school being related to achievement after controlling for individual students’ 
social class (Raudenbush and Bryk, 2002). HLM facilitates this type of multi-level analysis by specifying 
separate student and district equations. Following this model, an examination of district-level variables in 
Table 3.31 shows that average TASI dollars spent per student did not have a significant effect on district 
reading or mathematics scores. 
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Table 3.31. HLM Analyses of TAAS Mathematics and Reading TLI Scores of Cohort 2 
(2000-00) and Cohort 3 (2001-02) TASI Students 


Conditional Model 

Reading 

Mathematics 

Cohort 2 

Cohort 3 

Cohort 2 

Cohort 3 

Student Level Variables 

District Level Variables 

Gamma 

Coefficient/ 

(0 

Gamma 

Coefficient/ 

(0 

Gamma 

Coefficient/ 

it) 

Gamma 

Coefficient/ 

it) 

Intercept 


85.77 

(154.97**) 

86.46 

(216.54**) 

81.46 

(277.91**) 

82.97 

(293.56**) 


Economically 
disadvantaged context 

-0.009 

(0.47) 

-0.008 

(0.50) 

-0.005 

(0.48) 

-0.017 

(1.41) 


Achievement context 

-0.05 

(0.85) 

0.14 

(1.93) 

0.018 

(0.29) 

0.136 

(1.25) 


TASI per pupil 
expenditure 

-0.0003 

(0.72) 

-0.0003 

(0.42) 

0.00001 

(0.04) 

.00001 

(0.02) 


TASI program days 

0.0098 

(0.82) 

-0.004 

(0.41) 

-0.006 

(0.46) 

-.009 

(1-2) 

In Grade 6 


-7.35 

(17.44**) 

-4.88 

(9.38**) 

-2.87 

(6.90**) 

-1.84 

(6.33**) 

In Grade 7 


-2.37 

(5.58**) 

-1.18 

(2.98*) 

-1.01 

(4.07**) 

-0.69 

(2.45*) 

Female 


0.80 

(3.78**) 

0.26 

(1.83) 

0.57 

(3.14*) 

-0.17 

(-1.45) 

Minority status 


-2.00 

(5.34**) 

-0.94 

(4.64**) 

-1.08 

(4.40**) 

-0.93 

(5.94**) 

Economically 

disadvantaged 


-0.97 

(3.76**) 

-0.38 

(1.85) 

-0.66 

(3.30**) 

-0.29 

(1.79) 

Repeating current year 


5.79 

(5.22**) 

4.44 

(3.31**) 

4.81 

(6.06**) 

4.45 

(6.46**) 

School attendance rate 


0.21 

(5.72**) 

0.15 

(6.31**) 

0.22 

(15.4**) 

0.15 

(11.32**) 

TAAS pretest 


0.64 

(47.14**) 

0.66 

(55.30**) 

0.62 

(44.2**) 

0.702 

(54.02**) 

Days participated in TASI 


0.007 

(1.22) 

-0.002 

(0.59) 

0.004 

(1.00) 

0.002 

(0.63) 


Notes. In the student-level model, the intercept was specified as random and the independent variables were 
specified as fixed. *p < .01; **p < .001. 


Association of Student-and District-Levei Factors 
with Retention 

Relationships between TASI student and district characteristics and retention status (a binary outcome, 
retained or not retained) were analyzed using a hierarchical generalized linear model (HGLM) with a 
Bernoulli sampling model, a log odds or logit link function, and level- 1 (student) and level-2 (district) 
structural models identical to those in HLM. HGLM presents results for both unit-specific and 
population-average models. The unit-specific model holds constant school district attended, while the 
population- average model does not, but averages over all districts. Because the average log-odds of 
retention was found to vary significantly across districts, this variation should be controlled or held 
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constant. Consequently, only unit-specific results will be presented and discussed below. (Note, however, 
that results are similar for both models.) 

Included in separate HGLM analyses were students who participated in TASI in 2000-01 (cohort 2) and 
2001-02 (cohort 3). Cohort 2 included 8,071 TASI students from 42 school districts, whereas cohort 3 
included 11,779 students from 47 districts. These data were used to investigate the effect of TASI student 
and district characteristics on retention status (2001 retention status for cohort 2 and 2002 retention status 
for cohort 3). As previously defined, student characteristics included gender, economic status, prior 
retention, days participated in TASI, minority status, and grade group in the TASI participation year. 
School attendance in the year of TASI participation and the average of TAAS reading and mathematics 
TLI scores in the year of participation were also used as student-level predictors. District-level variables 
included the district TASI mean of the TAAS combined reading and mathematics score (achievement 
contextual effect), the district proportion of disadvantaged TASI students, and the district TASI per pupil 
expenditure for the appropriate cohort year (see Table 3.32). 


Table 3.32. Descriptive Statistics for Student Retention Data 


Variable Name 

N 

Mean 

SD 

Minimum 

Maximum 

Cohort 2 (2000-01) Student-Level Descriptive Statistics 

Grade 6 

8,071 

0.32 

0.47 

0 

1 

Grade 7 

8,071 

0.37 

0.48 

0 

1 

Gender 

8,071 

0.50 

0.50 

0 

1 

Minority 

8,071 

0.75 

0.43 

0 

1 

Economically disadvantaged 

8,071 

0.65 

0.48 

0 

1 

Prior retention 

8,071 

0.02 

0.13 

0 

1 

Retention 2001 

8,071 

0.03 

0.16 

0 

1 

School attendance 

8,071 

96.10 

4.10 

50.29 

100.0 

Combined 2000 TAAS score 

8,071 

79.67 

11.05 

14.50 

96.0 

Days participated in TASI 

8,071 

29.30 

31.96 

0 

174.0 

Cohort 2 (2000-01) District-Level Descriptive Statistics 

Economically disadvantaged 

42 

62.78 

18.96 

26.67 

97.22 

Reading and math achievement (2001) 

42 

79.01 

3.27 

71.63 

84.92 

TASI expenditure per pupil 

42 

727.52 

652.54 

42.87 

3,580.68 

Days TASI offered 

42 

50.36 

29.14 

0 

117.75 

Cohort 3 (2001-02) Student-Level Descriptive Statistics 

Grade 6 

11,779 

0.34 

0.47 

0 

1 

Grade 7 

11,779 

0.35 

0.48 

0 

1 

Gender 

11,779 

0.51 

0.50 

0 

1 

Minority 

11,779 

0.75 

0.43 

0 

1 

Economically disadvantaged 

11,779 

0.64 

0.48 

0 

1 

Prior retention 

11,779 

0.02 

0.12 

0 

1 

Retention 2001 

11,779 

0.02 

0.14 

0 

1 

School attendance 

11,779 

96.12 

4.02 

41.38 

100.0 

Combined 2000 TAAS score 

11,779 

82.25 

10.70 

29.50 

96.0 

Days participated in TASI 

11,779 

24.69 

26.26 

0 

189.0 

Cohort 3 (2001-02) District-Level Descriptive Statistics 

Economically disadvantaged 

47 

63.33 

19.57 

16.67 

98.96 

Reading and math achievement (2001) 

47 

80.97 

3.81 

71.54 

87.08 

TASI expenditure per pupil 

47 

405.33 

398.71 

450.55 

2,445.65 

Days TASI offered 

47 

49.81 

26.39 

5.18 

101.84 
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Student-Level Variables 

> After controlling for the effect of student-level characteristics (academic and social 
background), for cohort 3 students, more instructional days in TASI were associated with a 
marginally decreased chance of retention. 

> For students in both cohorts, an increase in a student’s school attendance rate decreased the 
chances of retention. 

As one might expect, being male and economically disadvantaged were associated with higher odds^ of 
retention (Table 3.33). Having higher TAAS scores, being female, and having been retained the year prior 
to TASI participation were also associated with lower odds of retention. More importantly, school 
attendance was related to lower retention. A 1 % increase in the attendance rate (above the average 
attendance rate) corresponded to a 0.13 reduction in the log-odds of retention in cohort 2 and to a 1.00 
reduction in cohort 3. These log-odds of retention corresponded to odds of 0.88 and 1.10, respectively. If, 
for example, there are two otherwise similar cohort 2 students but one has a 1 % higher attendance rate, 
the odds of retention for that student are 0.88 times the odds for the student without the additional 
attendance. In other words, the student with the 1 % higher attendance rate will have a smaller chance of 
being retained. This effect is much larger for cohort 3 students. 

For cohort 2 students, there was no significant association between the number of days in the TASI 
program and retention rate. However, there was a slight association for cohort 3 students. Other factors 
being equal, for cohort 3 students, each additional day of instruction in TASI resulted in a -0.007 
reduction in the log-odds of retention. This corresponds to odds of .99 and to a probability of .50. 

Consider the cohort 3 student who was in eighth grade, male, not economically disadvantaged, had no 
prior retention, and not minority. If this student also had average school attendance, average pre-TAAS 
scores, and attended a typical school, the probability this student would be retained is 0.0087. Attending 
TASI for 10 more days would reduce the retention probability to 0.0081. 

District-Level Variables 

> After controlling for the effect of student-level characteristics (academic and social 
background) and district social and academic contexts, there was no significant relationship 
between TASI dollars per pupil and retention rates. 

The only contextual variable to show any influence on district retention (Table 3.33) was the one 
measuring the percent of TASI students classified as economically disadvantaged. However, its influence 
was not in the expected direction. For cohort 3, schools with a higher percentage of economically 
disadvantaged students have a lower rate of retention. There was no association found between TASI 
program length and average dollars spent per student and retention. 


^ Odds are the probability that an event will occur divided by the probability that the event will not occur. If the 
weather forecast says that there is a 20% chance of rain, then there is an 80% chance that it will not rain, and the 
odds of rain are 0.20/0.80 or 0.25. If the forecast says that there is a 50% chance of rain, then there is also a 50% 
chance that it will not rain, and the odds of rain are 0.50/0.50 or 1 .00. If the forecast says that there is an 80% chance 
of rain, then there is a 20% chance that it will not rain, and the odds of rain are 0.80/0.20 or 4.00. 
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Table 3.33. HGLM Analyses of Retention Status of Cohort 2 (2000-01) 
and Cohort 3 (2001-02) TASI Students 


Conditional Model 

Cohort 2 

Cohort 3 

Student Level Variables 

District Level Variables 

Gamma 

Coefficient/ 

it) 

Gamma 

Coefficient/ 

it) 

Intercept 


-4.44 

(14.20**) 

-4.74 

(16.76**) 


Economically 
disadvantaged context 

-0.02 

(1.91") 

-0.03 

(2.82*) 


Achievement context 

0.05 

(0.74) 

0.07 

(1.34) 


TASI per pupil 
expenditure 

-0.0001 

(0.47) 

-0.00005 

(0.08) 


TASI program days 

-0.003 

(0.47) 

-0.0059 

(0.98) 

In Grade 6 


0.19 

(0.91) 

-0.12 

(0.66) 

In Grade 7 


.53 

(2.70*) 

.25 

(1.46) 

Female 


-0.63 

(4.06**) 

-0.38 

(2.68*) 

Minority status 


-0.44 

(2.14) 

-0.13 

(0.67) 

Economically 

disadvantaged 


0.73 

(3.69**) 

0.60 

(3.21*) 

Repeating current year 


-1.41 

(2.29*) 

-2.01 

(2.57*) 

School attendance rate 


-0.13 

(10.20**) 

-0.098 

(8.88**) 

TAAS pretest 


-0.08 

(12.83**) 

-0.074 

(14.04**) 

Days participated in TASI 


-0.002 

(0.61) 

-0.007 

(2.10") 


Note. Student-level predictors were specified as fixed with error terms constrained to 
be 0. Intercepts varied randomly. 

^ p < .05; *p < .01; **p < .001. 
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CONCLUSIONS AND IMPLICATIONS 


In 1999, the Texas Legislature created the Texas After-School Initiative. The program funds 
after- school programs targeting middle school students, ages 10 to 14, who are at risk of 
academic failure and/or at-risk of committing juvenile offenses. The purpose of this evaluation is 
to assess the effective use of TASI funds to create after-school programs supporting the 
academic success of targeted students in grades 6 through 8. This study includes 60 districts and 194 
campuses receiving both original and continuation TASI funding. Analyses examine results for grant 
dollars used during four school years (1999-00, 2000-01, 2001-02, and 2002-03). Conclusions and 
implications relative to the study’s research questions are summarized below. 


What are the characteristics of districts and students receiving TASI grants? 


> Characteristics of districts. Considering program goals, it was not surprising that TASI programs 
were more heavily concentrated in larger districts (more than 10,000 students). Smaller districts 
seldom had TASI programs. Compared to the state, TASI districts and campuses have larger 
proportions of Hispanic, LEP, and economically disadvantaged students, smaller proportions of 
White students, and nearly equal percentages of African Americans. 

^ Characteristics of students. During four program years (1999-00 to 2002-03), 101,771 unique 
students participated in the TASI program (based on an estimated count that adjusts for missing 
students). The number of students served in TASI programs increased during the four years of the 
grant from about 13,000 in spring 2000 to almost 32,000 in 2002-03. TASI participants included a 
nearly equal proportion of sixth, seventh, and eighth graders. 

More than three-fourths of TASI students are Hispanic (about 53%) and African American (22%) and 
about two-thirds are economically disadvantaged (59%). Even so, the majority of students served by 
TASI programs have never been retained in grade. Approximately 8% of TASI students had repeated 
one grade, and approximately 1% had multiple retentions. TASI programs targeted students at risk of 
failure — either those meeting the state Compensatory Education requirements or meeting other risk 
factors. Hispanic and African American students were more likely to be eligible for TASI services 
under Compensatory Education requirements, whereas White students were more likely to meet other 
risk factors or to have no at-risk characteristics. 


How did TASI resources supplement existing educational programs? 


> Program characteristics. TASI after-school programs were required to directly address the needs 
of students by incorporating three components: (a) an academic-based curriculum linked to the Texas 
Essential Knowledge and Skills (TEKS), (b) a character/citizenship education component, and (c) a 
plan for parental and/or mentor involvement. Overall, TASI districts dedicated the greatest percentage 
of after-school time to the academic-based component. Eor more than three-quarters of programs, 

41% or more of time was spent on academics. 

TASI programs typically offered about four instructional days per week, with nearly two program 
hours each day. The mean number of instructional days varied across years from about 49 to 58. 
Although after-school programs used a variety of grouping configurations, programs most frequently 
relied on medium-sized group activities (7 to 19 students) for both academic-based and 
cbaracter/citizenship program components. 

^ Academic component. Instructional technology, as well as more traditional approaches, were 
commonly used in TASI academic components. Lightspan was the most commonly used program 
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throughout the grant’s duration. General use of technology in academic activities (such as word 
processing and Internet) was also prevalent. Time dedicated to technology, however, declined 
somewhat over the grant period. On the contrary, tutoring remained one of the most commonly used 
strategies throughout the four-year period. 

> Character education/citizenship component. Districts used a comhination of commercial 
programs {Voyager, ROPES, Character Counts, etc.), external supports (guest speakers, field trips, 
community service, etc.), and other activities (athletics, fine arts, etc.) as part of the TASI character 
education/citizenship component. 

> Parent and mentor component. Traditional means of communicating with parents (meetings, 
mail, telephone) and involving parents (training, volunteering) were most commonly used in TASI 
programs. Parent involvement most commonly meant attending a parent night or serving as a helper. 
Mentors most often served as tutors or guest speakers. 


What was the effect of grant resources on targeted students? 


To gauge the effectiveness of the TASI program in improving student success, we examine TASI 
students’ core-suhject area course passing rates and discipline referrals. In addition, comparisons are 
made between program participants and a comparable group of non- served students for other measures, 
including attendance rates, performance on state-level assessments, and retention rates. 

> Core-subject course passing rates. The majority of TASI students (84% to 89%) passed core 
content-area courses. Course passing rate were slightly lower for mathematics (84% to 86%). 

Students meeting state Compensatory Education requirements or having other risk factors had lower 
passing rates (up to 10 percentage points less than more advantaged peers). 

> Discipline referrals. Discipline problems are an issue for some TASI students. Nearly one-fifth of 
students (17% to 20%) had four or more office referrals, and a small percentage of students were 
referred to alternative education programs (about 5%) or juvenile justice programs (less than 1%). 
Since disciplinary referrals remained relatively stable across program years, there appeared to be no 
association between TASI and the improvement of student discipline. 

> Attendance rates. In general, TASI students’ attendance rates did not improve over time. However, 
attendance rates for TASI students during the program year (about 96%) were consistently higher 
than rates for a comparison group of non-TASI students (about 95%). TASI students who are in grade 
for the first-time have higher attendance rates than students repeating a grade level for both TASI and 
non-TASI comparison groups. Although attendance rates for repeating TASI students declined over 
time, a slightly positive change was observed during the TASI program implementation year for the 
last three cohorts. 

> State-level assessments. TASI students had lower TAAS passing rates for both reading and 
mathematics compared to non-TASI students, but the achievement gap between groups narrowed 
slightly for student cohort 1 to 3. Despite apparent progress, the achievement gap between TASI and 
non-TASI students increased in both reading and mathematics for cohort 4 students who completed 
the TAKS assessments. For the small number of students repeating their grade level, the TAAS 
passing rate gap was narrowed between TASI and non-TASI student cohorts. In fact, TASI students 
in cohort 3 passed TAAS reading at a higher rate than non-TASI students. Moreover, TASI students 
maintained their advantage over non-TASI students on the TAKS reading assessment and nearly 
equaled comparison-group students on TAKS mathematics. 

Grade-level results show that TASI students generally had lower TAAS passing rates for both 
reading and mathematics than non-TASI comparison group students across all grade levels (6, 7, 
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and 8). Consistent with results for all students, the achievement gap between TASI and non-TASI 
students narrowed slightly for TAAS. TAKS results for cohort 4 students are less promising, 
especially for math. TAKS passing rates were lower than TAAS scores and the achievement gap 
between TASI and non-TASI students increased. 

^ Retention. For TASI students, retention rates declined across cohorts (3% to 2.2%), and in cohort 3, 
TASI students had slightly lower retention rates slightly lower than a comparison group of non-TASI 
students (2.2% compared to 2.5%). For the small number of students repeating their grade level, 
across-cohort trends show that TASI students had slightly lower retention rates (3%, 2.9%, 2.2%) 
than non-TASI students over time (2.5%, 3%, 2.5%). Although student retention varies by grade 
level, TASI students’ retention rates declined across all cohorts for all grade levels. By cohort 3, 
seventh- and eighth-grade TASI students had lower retention rates than non-TASI students. 


What program elements are associated with student outcomes? 


To further explore the association between TASI student and district characteristics and TAAS reading 
and Mathematics TLI scores, researchers used hierarchical linear modeling (HLM). Separate HLM 
analyses were conducted using TASI participants in 2000-01 (cohort 2) and 2001-02 (cohort 3). Cohort 2 
included 7,601 students in 42 school districts and cohort 3 included 11,172 students in 45 districts. These 
students participated in extended-day programs in their respective TASI year. Separate HLM analyses 
were also conducted for retention. Cohort 2 included 8,071 students in 42 districts and cohort 3 included 
11,779 students in 47 districts. 

Student-Level Factors and Outcomes 

> After controlling for the effect of student-level characteristics (academic and social background), 
there was no positive relationship between the number of instructional days students spend in TASI 
(up to 189 days) and TAAS scores. This suggests that the academic component of the TASI program 
was not optimally effective in improving student academic performance. In contrast to achievement, 
more instructional days in TASI were associated with a marginally decreased chance of retention for 
cohort 2 students (2001-02). Outcomes for HLM analyses are consistent with findings for TASI and 
non-TASI comparison groups. TASI appears to have had little or no impact on achievement but may 
have been somewhat effective in reducing student retention. 

> A TASI student’s school attendance rate was a significant predictor of academic performance. Higher 
school attendance rates were associated with higher TAAS reading and mathematics scores. In 
addition, for otherwise similar students, an increase in the school attendance rate decreased the 
chances of retention. Clearly, a student’s school attendance rate was a significant predictor of valued 
outcomes (achievement and promotion) — thus, efforts aimed at improving a student’s day-to-day 
presence in school seems to have a greater probability of improving academic performance than 
participation in an after- school program. 

District-Level Factors and Outcomes 

> Consistent with findings for the Optional Extended Year Program (OEYP), there was no compelling 
evidence that increasing per-pupil TASI expenditures improved academic performance or reduced the 
probability of student retention. There was no significant relationship between TASI dollars per pupil 
and TAAS reading and mathematics scores. Eikewise, there was no significant relationship between 
TASI dollars per pupil and retention rates. Yet again, evidence suggests that how educational dollars 
are used makes a difference. Resources must be invested in programs that have a research-based link 
to valued student outcomes. Simply adding more dollars for programs appears unlikely to achieve the 
desired effect. 
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What are the implications for addressing the needs of students at risk? 


> After-school programs, as they are currently designed, appear only marginally successful In 
Improving the academic performance of the majority of student participants. For students in 
their grade for the first time (the majority of TASI participants), program participation had no 
discemahle relationship to improved school attendance rates and only a modest correlation with 
increased TAAS scores. Despite some TAAS gains in reading and mathematics, the majority of TASI 
students lost ground compared to their non-TASI counterparts on the TAKS assessments, especially 
in mathematics. 

> After-school programs may provide the greatest benefit for students who have been retained 
In grade. There was a stronger relationship between TASI participation and both attendance and 
TAAS scores for students repeating in a grade level. A slightly positive change was observed for 
student attendance during the TASI program year. Moreover, for a small number of students 
repeating their grade level, the TAAS passing rate gap was narrowed between TASI and non-TASI 
students. Further, TASI students in cohort 4 who were repeating a grade maintained their passing rate 
advantage in TAKS reading over their non-TASI counterparts. However, for mathematics, both 
groups had extremely low passing rates. 

> Reducing student retention through participation In an after-school program does not 
necessarily translate Into Improved academic achievement. There was some indication that 
retention rates were reduced for TASI participants across all categories of students. However, simply 
preventing student retention did not ensure an increase in knowledge and skills as measured by state 
assessments. Effective programs are those with a strong correlation between reduced student retention 
and improved academic outcomes. 

> There Is a need for a closer examination of the cost-effectiveness of after-school programs. 

Although we compiled a great deal of information on after-school programs, program effectiveness, 
and especially cost-effectiveness, remains uncertain. Some trends have been revealed, but a more in- 
depth examination of specific programs is needed in order to understand what programs work, for 
whom, and under what circumstances. One way to approach such a study would be to use the 
databases created for this research study to identify districts that had successful programs. Success 
would be defined through a “value-added” approach. HEM analyses could identify programs that, 
after controlling for student- and school-level characteristics, contribute to improved student 
academic achievement. 
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Chapter 4 

NINTH GRADE SUCCESS INITIATIVE (NGSI) 


INTRODUCTION 

From 1999 to 2002, the state of Texas appropriated a total of $170 million for the Basic Skills Program 
for High School Students — also known as the Ninth Grade Success Initiative (NGSI) — to support school 
districts’ efforts to help ninth graders stay in school and succeed academically. The goal of NGSI was to 
increase graduation rates in Texas public schools hy reducing the number of students who were retained 
in or dropped out of the ninth grade. 

In 1999, the 76th Texas Legislature appropriated the first installment of $85 million for the 2000-01 
biennium for NGSI. The Texas Education Agency (TEA), via a competitive grant process, awarded NGSI 
grants to a total of 234 school districts and education consortiums in spring 2000. Nine more districts 
received funding beginning in the fall of 2000. In 2001, the 77th Texas Eegislature appropriated another 
$85 million to continue the NGSI for the 2002-03 biennium. Renewal grants supported 226 of the original 
districts, all of which continued their NGSI activities for the 2001-02 and 2002-03 school years. 

Eunds went toward expanding or enhancing existing programs, or creating new programs to increase 
academic performance and attendance rates and reduce dropout rates. NGSI programs had to serve 
(a) ninth graders who had not earned — or were not likely to earn — sufficient credit to advance to tenth 
grade and who failed to meet minimum skill levels, or (b) eighth graders who were being promoted to 
ninth grade but were considered academically at risk. Programs had to emphasize basic skills in core 
curricular areas and provide targeted students with opportunities to build credits toward graduation. NGSI 
was expected to achieve four major objectives: (1) decrease the rate of retention in ninth grade, (2) reduce 
the number of ninth grade dropouts, (3) increase attendance rates in ninth grade, and (4) support 
successful performance on the state’s assessments, including the exit-level Texas Assessment of 
Academic Skills (TAAS). 

Organization of the Chapter 

Sections to follow include a brief literature review on the ninth-grade problem and an overview of this 
study’s methodology. In addition, NGSI program findings are presented for the following topics: 

(a) NGSI grantees and their students; (b) NGSI program activities; (c) student outcomes for course 
grades, attendance, state-level assessments, and retention; (d) association of student- and district-level 
factors with academic achievement; (e) association of student- and district-level factors with retention; 
and (f) conclusions and implications. 


THE NINTH GRADE PROBLEM 

Ninth graders in America’s schools are at particular risk of failure. More students repeat the ninth grade 
than at any other time during their schooling career and this leads directly to an increase in later school 
dropouts (Barro & Kolstad, 1987). Texas has identified a set of characteristics related to grade retention 
among students. Students who are male, members of minority groups, limited English proficient, 
receiving special education services, or over-age for their grade are retained more often than other 
students (TEA, 1999). As Eigure 4.1 illustrates, during the five years prior to the initiation of the NGSI 
programs, ninth-grade retention rates averaged 17.5%. In comparison, eighth-grade retention rates 
averaged 2.0% and tenth grade 7.7%. Nationally, the rate at which ninth graders fail to advance to 10th 
grade has tripled in the last 30 years, a trend particularly troublesome for students deemed “at-risk” since 
60% will not graduate with their class (Green & Scott, 1995). 


75 




Explanations for the difficulty some students encounter in ninth grade focus variously on the role of the 
students’ background (including weak mathematics and reading comprehension skills), teachers who lack 
the knowledge or instructional strategies required to instruct teenagers whose basic skills need 
improvement, or teachers who take a “sink or swim” approach with their students (Roderick & Cambum 
1999; Balfanz, McPartland, & Shaw, 2002). Several researchers have found a negative association 
between school size and student achievement, with this result independent of school setting (rural or 
urban) and staff and student characteristics (Eberts, Kehoe, & Stone 1982; Eowler & Walberg, 1991). 
Other studies indicate that the middle-to-high school transition is at the root of academic distress in ninth 
grade. A new school environment presents stresses, particularly when several middle schools “feed” into 
a single high school (Schiller, 1999). Students enter a larger, more impersonal and complex school 
structure in which there is more competition and a greater orientation to grades and academic 
achievement (Eccles, Midgley, & Adler, 1984; Roderick & Camburn, 1999). 



□ 8th grade □ 9th grade ■ 1 0th grade 


Figure 4.1. Texas retention rates for students in grades 8, 9, and 10 by year. 

Sottrce; Texas Education Agency, 1999. 

Evidence from research offers potential solutions to the ninth-grade problem. Schools with a more 
extensive transition program had lower ninth-grade dropout and retention rates (Hertzog & Morgan, 

1999). Results for a comprehensive school reform program, the Talent Development High Schools, also 
suggest that freshman failure is responsive to changes in school organization and curriculum (Letgers et 
al. 2002). Throughout the country, high schools are experimenting with a variety of strategies to address 
the ninth grade problem. The most successful results appear to include a combination of strategies, 
ongoing support for students throughout the ninth-grade year, and regular communication with and 
support from parents. 

METHODOLOGY 

This report relies on a combination of quantitative and qualitative data and methods as described below. 
Analyses include 226 school districts receiving both original and continuation NGSI funding. Districts 
received an original two-year grant for five semesters (spring 2000 through summer 2001) and a two-year 
continuation grant for six semesters (fall 2001 through summer 2003). NGSI funding and programming 
began at mid-semester in spring 2000, so program data for the term are excluded. Outcome data for 
students participating in this first semester, however, serve as a baseline to measure student gains. 
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Data Sources 


The primary data sources include NGSI program and activity reports submitted by districts and student 
demographic and performance data collected from the TEA. 

Program and activity reports. NGSI grant recipients were required to submit program and activity 
reports to the TEA after each semester in which they served targeted students. Across four program years 
(1999-00 through 2002-03), districts were asked to submit two reports after each of 11 semesters in which 
they served students. The program report requested district-level information, such as general program 
information, activities supporting credit recovery and basic skills, dissemination activities, staff 
participation and involvement, professional development, and district opinions regarding the most 
successful components. Districts also submitted activity reports with student-level data each semester. 
Activity reports provided information in six areas for each NGSI program participant: student 
demographic information, student eligibility, school attendance, retention and promotion, activities 
engaged in, and student performance. As with the program report, all information is self-reported. During 
the course of the 1 1 program terms, report format changes resulted in some data discontinuity. 

Student demographic and performance data. Researchers gathered other student-level data from the 
Texas Public Education Information Management System (PEIMS) and the Texas Academic Excellence 
Indicator System (AEIS). Student-level data obtained from activity reports were matched to PEIMS and 
AEIS data to create a set of master databases. Elements in the databases included student demographic 
information such as ethnicity, gender, limited English proficiency (LEP) status, and grade level. Outcome 
data related to the TAAS, Texas Assessment of Knowledge and Skills (TAKS), attendance, and retention. 

Limitations 

Data accuracy. District compliance with reporting requirements varied throughout the initiative. Eor the 
first term (spring 2000), only two out of three districts submitted activity reports (Table 4.1). This was at 
least partially because the first semester was a start-up period, and many district programs either were not 
functioning or were not fully functioning. During the first full-implementation term (summer 2000), 
activity report submissions increased. Submission rates varied from 88% (spring 2001) to 59% (summer 
2003). Submission rates for program reports ranged from 82% (fall 2000) to 36% (summer 2002). 


Table 4.1. District Program and Activity Report Submission 



Program Reports 
(District Level) 

Activity Reports 
(Student Level) 

N 

% 

N 

% 1 

Regular Term 

Spring 2000 



149 

66 

Fall 2000 

185 

82 

166 

73 

Spring 2001 

183 

81 

200 

88 

Fall 2001 

149 

66 

175 

77 

Spring 2002 

134 

59 

180 

80 

Fall 2002 

91 

40 

173 

77 

Spring 2003 

97 

43 

154 

68 

Summer Terms 

Summer 2000 

159 

70 

174 

77 

Summer 2001 

169 

75 

179 

79 

Summer 2002 

81 

36 

136 

60 

Summer 2003 

97 

43 

133 

59 


Note. N= 226 participating districts. 
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Varying report submission rates are problematic for two reasons. First, it is possible that the districts 
included in the analysis, by virtue of submitting reports, are not representative of all the districts funded 
under NGSI. Second, when comparing results from year to year, it is possible that differential response 
rates across terms lead to incorrect conclusions, as the districts are not the same from one term to the next. 
Table 4.2 examines district submissions across terms. Of the 226 original and continuing districts, only 
18% submitted activity reports for all 11 terms, but 76% of the districts submitted reports for 7 or more of 
terms. This group of 172 districts forms the core NGSI group included in analyses — thus, results are most 
representative of these districts. 


Table 4.2. Number of Student Activity Reports Submitted by Districts 


Number of Reports 
Submitted 

Number of 
Districts 

Percent of 
Districts 

Cumulative 

Percent 

Eleven 

41 

18.1 

18.1 

Ten 

33 

14.6 

32.7 

Nine 

33 

14.6 

47.3 

Eight 

39 

17.3 

64.6 

Seven 

27 

11.9 

76.5 

Six 

20 

8.8 

85.4 

Eive 

11 

4.9 

90.3 

Eour 

9 

4.0 

94.2 

Three 

7 

3.1 

97.3 

Two 

3 

1.3 

98.7 

One 

2 

0.9 

99.6 

Never submitted a report 

1 

0.4 

100.0 

Total 

226 

100.0 



Estimating student participation. Student data for each term were combined to estimate the total 
number of unique students served during the four-year grant period (see Table 4.3). By replacing missing 
term data with a weighted district term average, the number of unique students served during four NGSI 
program years was estimated at 389,834 students. (See Appendix B for a full explanation.) The estimated 
number of students for each program year ranged from 45,867 (1999-00) to 121,364 (2001-02). 


Table 4.3. Number of Students Served in NGSI by School Year 


Year 

Term 

Duplicated Student Count 

Unique Student Count 

Activity 
Report N 

Estimated N 

Activity 
Report N 

Estimated N 

1999-2000 

Spring 2000 

16,134 

32,535 

25,532 

45,867 

Summer 2000 

13,558 

19,508 

2000-2001 

Pall 2000 

47,354 

59,580 

78,640 

104,516 

Spring 2001 

61,951 

70,922 

Summer 2001 

18,186 

24,144 

2001-2002 

Pall 2001 

50,621 

63,376 

75,181 

121,364 

Spring 2002 

58,397 

70,392 

Summer 2002 

13,412 

27,171 

2002-2003 

Pall 2002 

52,400 

65,897 

79,558 

118,087 

Spring 2003 

55,836 

73,847 

Summer 2003 

20,775 

31,607 

Total 


408,624 

538,979 

258,911 

389,834 
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NGSI GRANTEES AND THEIR STUDENTS 


This section presents findings on the characteristics of districts receiving NGSI grants as well as the 
characteristics of students participating in the NGSI. 

Location of Districts Receiving Grants 

> The largest concentration of NGSI grantees was in ESC regions 1 (Edinburg) and 
4 (Houston). 

The distribution of NGSI grantees corresponds roughly to Texas population centers, with densely 
populated regions (e.g., Education Service Center [ESC] 4, Houston and ESC 20, San Antonio), East 
Texas regions (ESC 6, Huntsville and ESC 7, Kilgore), and a Rio Grande Valley region (ESC 1, 
Edinhurg) having large numbers of programs. 


Table 4.4. NGSI Grantees by Education Service Center Region 


ESC Region 

ESC 

Location 

Number of 
Grantees 

Percent of 
Grantees 

Statewide 

Distribution 

1 

Edinburg 

24 

10.6 

4.1 

2 

Corpus Christi 

10 

4.4 

3.9 

3 

Victoria 

4 

1.8 

3.3 

4 

Houston 

26 

11.5 

8.3 

5 

Beaumont 

7 

3.1 

2.9 

6 

Huntsville 

18 

8.0 

4.9 

7 

Kilgore 

20 

00 

00 

8.4 

8 

Mt. Pleasant 

2 

0.9 

3.9 

9 

Wichita Falls 

3 

1.3 

3.3 

10 

Richardson 

16 

6.2 

9.2 

11 

Ft. Worth 

14 

6.2 

7.2 

12 

Waco 

16 

7.1 

7.0 

13 

Austin 

17 

7.5 

5.8 

14 

Abilene 

3 

1.3 

3.6 

15 

San Angelo 

2 

0.9 

3.6 

16 

Amarillo 

7 

3.1 

5.3 

17 

Lubbock 

5 

2.2 

5.1 

18 

Midland 

5 

2.2 

2.9 

19 

El Paso 

8 

3.5 

1.3 

20 

San Antonio 

19 

8.4 



Note. N = 226 NGSI districts. 


Characteristics of Districts Receiving Grants 

> Although NGSI districts vary by size, more than half (58%) of grant recipients are midsize to 
very-iarge districts. 

NGSI district size distributions differ from the state, in which a majority of districts have very small 
enrollments (60% with fewer than 1,000 students). In contrast, the majority of NGSI districts are mid-size 
(29%), large (16%), or very large (13%). (See Table 4.5.) 
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Table 4.5. NGSI Grantees by District Size Categories 


District Size: Student Enrollment 

Number of 
Districts 

Percent of 
Districts 

State 

Average 

Very small: Fewer than 1,000 

32 

14.2 

59.6 

Small: 1,000 to 3,000 

63 

27.9 


Midsize: 3,001 to 10,000 

66 

29.2 

12.9 

Large: 10,001 to 25,000 

36 

15.9 

3.8 

Very large: More than 25,000 

29 

12.8 

3.2 


Source. TEA AEIS database 2002-03. N = 226 NGSI districts 


> High schools in NGSI districts have a slightly greater proportion of minority, limited English 
proficient, and economically disadvantaged students. 

Compared to the state, high schools in NGSI districts have slightly higher percentages of Hispanic and 
African American, limited English proficient, and economically disadvantaged students (Table 4.6). 


Table 4.6. Student Demographics for NGSI Districts (Percent) 


Student Demographics 

High Schools in 
NGSI Districts 

State High 
Schools 

White 

35.6 

44.8 

Hispanic 

44.7 

37.8 

African American 

15.9 


Economically disadvantaged 

43.5 

38.5 

Limited English proficient 

8.3 

6.6 

Special education 

12.0 

12.3 


Source. TEA AEIS database 2002-03. 


Characteristics of Students Participating in NGSI 

> The number of students participating in NGSI program activities varied by year, with 
substantially fewer students served during the summer terms. 

In 2000-01, the first full program year, 210 districts reported participation hy 70,680 students during 
regular terms (fall and spring). Adjusting this number for missing data, it is estimated that the actual 
number of student participants during the regular school year was 90,519 (see Table 4.7). The largest 
number of estimated students (106,325) participated during the last program year (2002-03). The number 
of students enrolled in summer school climbed steadily during the grant period. 


Table 4.7. NGSI Program Participation 



Regular Terms 

Summer Terms 

Spring 

2000 

2000-01 

2001-02 

2002-03 

2000 

2001 

2002 

2003 

Districts submitting a report 

149 

210 

196 

199 

174 

179 

136 

133 

Number of students^ 

15,968 

70,680 

69,927 

76,323 

13,558 

18,186 

13,412 

20,775 

Number of students per program 
(mean) 

108 

336 

357 

383 

78 

102 

99 

156 

Estimated student enrollment’’ 

32,535 

90,519 

91,403 

106,325 

19,508 

24,144 

27,171 

31,607 


“ Calculated as the number of unique students across the identified time period for which data are available. 
Estimated number of students served. See methodology in Appendix B. 
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> The majority of students served in NGSi programs were ninth-grade students at-risk of not 
earning sufficient credits to advance to tenth grade. 

NGSI summer programs served an increasing percentage of ninth graders at risk of not earning sufficient 
credits to advance to the next grade level, while percentages of newly promoted ninth graders declined 
steadily across the summer terms (Table 4.8). 


Table 4.8. Student Eligibility by Term (Percent) 



Regular Terms 

Summer Terms 

Spring 

2000 

2000- 

01 

2001- 

02 

2002- 

OS 

2000 

2001 

2002 

2003 

Newly promoted but lacking basic skills 

— 

— 

— 

— 

32.7 

19.0 

10.2 

8.6 

At-risk of not earning sufficient credits 

82.2 

81.0 

87.8 

83.9 

41.9 

67.1 

70.0 

75.6 

Did not earn sufficient credits for 
promotion 

17.8 

19.0 

12.2 

16.1 

25.4 

13.9 

19.8 

15.8 


Note. Students could be classified as newly promoted only during the summer terms. 


> About three-fourths of NGSI students are Hispanic and African American. 

Across years, a consistently high percentage of Hispanic students participated in NGSI activities. In 
contrast, African American students participated at higher rates during the summer terms. 


Table 4.9. Distribution of Student Race/Ethnicity by Semester 
and Student Eligibility (Percent) 



Regular Terms 

Summer Terms 

Spring 

2000 

2000-01 

2001-02 

2002-03 

2000 

2001 

2002 

2003 

All NGSI Participants 

Hispanic 

46.1 

58.5 

60.5 

57.2 

53.8 

57.9 

52.9 

60.2 

White 

31.7 

24.0 

23.0 

25.7 

22.1 

21.1 

22.0 

21.5 

African American 

22.2 

15.7 

15.1 

15.4 

23.1 

19.7 

23.7 

16.9 

Female 

42.9 

45.9 

45.7 

45.9 

43.6 

44.4 

44.5 

44.1 

Male 

57.1 

54.1 

54.3 

54.1 

56.4 

55.6 

55.5 

55.9 

At-risk of not earning sufficient credits 

Hispanic 

45.4 

57.9 

61.2 

57.1 

55.8 

58.6 

55.1 

59.9 

White 

34.1 

25.5 

22.7 

26.0 

23.5 

20.2 

24.2 

22.2 

African American 

18.8 

14.8 

14.7 

15.1 

19.6 

19.9 

19.3 

16.4 

Female 

43.5 

47.2 

46.2 

46.6 

43.8 

43.9 

43.8 

44.0 

Male 

56.5 

52.8 

53.8 

53.4 

56.2 

56.1 

56.2 

56.0 

Did not earn sufficient credits for promotion 

Hispanic 

51.5 

60.3 

62.2 

63.2 

56.8 

58.1 

54.8 

65.2 

White 

22.6 

17.3 

19.1 

20.4 

18.7 

16.9 

15.0 

15.7 

African American 

24.2 

20.8 

16.6 

14.9 

23.4 

23.3 

28.3 

17.7 

Female 

39.2 

40.4 

41.1 

40.1 

39.3 

40.9 

41.4 

42.6 

Male 

60.7 

59.6 

58.9 

59.9 

60.7 

59.1 

58.6 

57.4 

Newly promoted but lacking basic skills 

Hispanic 

— 

— 

— 

— 

42.5 

50.3 

43.9 

50.9 

White 

— 

— 

— 

— 

26.1 

27.1 

28.9 

24.5 

African American 

— 

— 

— 

— 

30.4 

20.7 

26.1 

22.7 

Female 

— 

— 

— 

— 

46.8 

48.6 

49.8 

45.2 

Male 

- 

- 

- 

- 

53.2 

51.4 

50.2 

54.8 


Note. Students could be classified as newly promoted only during the summer semesters. Between 0.2% and 
2.1% lacked ethnicity data and between 0.1% and 4.3% lacked gender data. 
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Ethnic distributions also varied by student eligibility categories, with smaller percentages of White 
students comprising the group of students who did not earn sufficient credits for promotion. Greater 
percentages of male students were NGSI participants, especially among students who did not earn 
sufficient credits for promotion (Table 4.9). 

> The majority of students served by NGSI programs were in the ninth grade for the first time 
(80% or more each term). 

As noted in Table 4.10, about 80% to 88% of NGSI participants were first-time ninth graders, and 
between 10% and 19% of students had repeated ninth grade at least once. 


Table 4.10. Prior Student Retention in Grade 9 by Semester (Percent) 



Regular Terms 

Summer Terms 

Spring 

2000 

2000-01 

2001-02 

2002-03 

2000 

2001 

2002 

2003 

First time in 9th 

82.2 

81.0 

87.8 

83.9 

80.5 

83.0 

* 

84.2 

Second time in 9th 

16.5 

16.2 

10.4 

14.1 

18.8 

15.4 

* 

14.3 

Third time in 9th 

1.1 

2.4 

1.6 

1.6 

0.7 

1.4 

* 

1.4 

Fourth time in 9th 

0.1 

0.4 

0.2 

0.4 

0.1 

0.2 

* 

0.1 


Note. Between 6.5% and 28.4% of students lacked prior grade 9 retention data. 
*Data for Summer 2002 excluded due to data quality issues. 


NGSI PROGRAM ACTIVITIES 

Information on NGSI program activities come from two sources. First, each district completed program 
reports describing staff participation and involvement and indicating their approach to credit recovery and 
basic skill improvement by selecting from a list of methods (e.g., tutoring, instructional technology). 
Second, districts submitted activity reports identifying how each student accrued course credits (e.g., 
regular classroom instruction, credit accrual) and the kinds of activities experienced (e.g., tutoring, 
counseling). 

Staff Participation and Invoivement 

> On average, NGSI programs had larger staff, more volunteers, and higher student-to-teacher 
ratios for fall and spring semesters compared to summer terms. 

Teachers, instructional aides, and volunteers assisted in the implementation of NGSI activities each 
semester. Information on staff participation and involvement is displayed in Table 4.11. In fall and spring 
semesters, NGSI programs averaged approximately 10 to 15 teachers, whereas summer programs had 
only 7 to 10 teachers. Because NGSI programs served fewer students in summer terms, mean student-to- 
teacher ratios in the summer (10.2 to 13.0) were substantially lower compared to regular terms (23.2 to 
27.4). Overall, programs averaged 6 to 9 paid teachers, 1 to 8 volunteer teachers, and 1 to 2 instructional 
aides each term. 
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Table 4.11. Staff Participation and Involvement, Average Number per Program 



Regular Terms 

Summer Terms 

2000-01 

2001-02 

Fall 2002 

2000 

2001 

2002 

Teachers 

All teachers 

14.9 

11.6 

10.2 

9.4 

9.8 

7.4 

Paid teachers (full or partial) 

7.0 

6.8 

6.4 

7.4 

9.1 

7.3 

Volunteer teachers 

7.9 

4.9 

3.8 

2.1 

1.2 

0.9 

Student-to-teacher ratio 

23.2 

24.5 

27.4 

11.7 

10.2 

13.0 

Instructional aides and volunteers 

Instructional aides 

1.6 

1.5 

1.3 

1.1 

1.5 

2.0 

Volunteers per program 

2.3 

2.2 

1.0 

0.8 

0.5 

0.6 


Note. The number of NGSI programs included in analyses varied by staff category. 
Source: NGSI program reports. 


Activities Supporting Credit Recovery and Basic Skiiis 

NGSI programs reported the extent to which they used the following activities to serve ninth graders at 
risk academically: 

• Tutoring — Additional help heyond regular classroom instruction; 

• Instructional technology — The use of technology for the purpose of gaining course credit or basic 
skills; 

• Individual instruction — One-on-one teaching and learning; 

• Group instruction — Instruction in the regular classroom; 

• Counseling — Services that help students match their interests with educational pursuits toward 
graduation; 

• Mentoring — Role models working with students for the purpose of improving their academic, 
decision-making, and problem solving skills; 

• Teaming — Grouping of students and teachers into smaller groups for the intent of enhanced 
instruction; 

• Trailer courses — Courses offered during a semester that allow students to recover previous 
semester course credit; 

• Credit by exam — Course credit through a comprehensive exam; 

• Open entry/exit courses — Alternative means of learning and gaining class credit in non-sequential 
subjects through courses with flexible entry and exit opportunities; 

• Correspondence course — Course credit through another learning institution; and 

• Distance learning — Learning in which some materials and/or participants are not local. 

> NGSI programs typically used several activities to serve at-risk ninth graders. Tutoring, 
instructional technology, individual instruction, group instruction, and counseling were 
reported most often. 

As Table 4.12 indicates, during the regular terms, programs relied on tutoring most often, followed by 
instructional technology for basic skills remediation or credit accrual. Individual instruction, group 
instruction in the regular classroom, and counseling also were commonly used. In contrast, summer 
programs most frequently used group instruction in the regular classroom, instructional technology, and 
individual instruction. There appeared to be less emphasis on tutoring and counseling during summer 
terms; however, lower student-to-teacher ratios during summer terms may have allowed teachers more 
time for individualized student attention. 
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Table 4.12. NGSI Program Activities (Percent of Programs) 



Regular Terms 

Summer Terms 

Activity 

2000-01 

2001-02 

Fall 2002 




Tutoring 

89.80 

84.10 

87.90 

52.20 

57.50 


Instractional technology 

78.0© 

74.6© 

74.7© 

66.7© 

66.3© 

66.7© 

Individual instruction 

71.3© 

72.4© 

71.4© 

56.0© 

60.6© 

65.4© 

Group instruction 

70.70 

67.50 

61.50 

70.40 

66.70 

69.10 

Counseling 

68.2© 

65.7© 

61.50 

47.8© 

51.3© 

46.9© 

Mentoring 

47.2 

47.4 

45.1 

29.6 

28.7 

33.3 

Teaming 

39.8 

29.3 

25.3 

23.3 

20.6 

21.0 

Trailer courses 

25.7 

19.8 

20.9 

35.8 

25.0 

22.2 

Credit by exam 

19.3 

16.2 

12.1 

17.6 

17.0 

17.3 

Open entry/open exit 

24.4 

20.9 

22.0 

18.9 

26.4 

19.8 

Correspondence course 

4.9 

2.5 

1.1 

4.4 

2.5 

2.5 

Distance learning 

2.7 

2.8 

2.2 

1.9 

1.3 

1.2 


Note. Numeric symbols (e.g., O) indicate the most frequently identified activities by rank order. 
Source: NGSI program reports. 


Course Credit Accrual and Instructional Methods 

Information from student-level activity reports described the coursework delivery method used for NGSI 
students. Students could accrue credit in four ways: regular classroom instruction, computer-aided credit 
accrual, trailer courses, and credit by exam (see Table 4.13). Some reports (specifically the first four and 
last two terms) also provided information on student participation in tutoring, counseling, teaming, and 
mentoring (see Table 4.14). 

> During the regular school year, students accrued course credit primarily through regular 
classroom instruction, but repeat ninth graders were more likely to accrue credit through 
computer-aided instruction. 

During the regular fall and spring terms, 61% to 90 % of first-time ninth graders at risk of not earning 
credits accrued credit via regular classroom instruction, compared to 43% to 66% of repeat ninth graders. 
Instead, 20% to 30% of repeat ninth graders accrued credit via computer-aided instruction. During 
summer terms, the proportion of first-time ninth graders accruing credit through computer-aided 
instruction increased substantially, such that participation rates were equivalent to repeat ninth graders. 
Across four grant years, reliance on computer-assisted credit accrual decreased during regular terms but 
increased in summer terms (Table 4.13). 
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Table 4.13. NGSI Course Credit Accrual Methods (Percent of Students) 



Regular Terms 

Summer Terms 

2000-01 

2001-02 

2002-03 

2000 

2001 

2002 

All NGSI Participants 

Regular classroom instruction 

57.1 

66.5 

86.9 

53.0 

63.9 

72.0 

Computer-aided credit accrual 

12.9 

12.1 

9.3 

14.0 

14.3 

30.3 

Trailer course 

3.6 

3.3 

0.1 

6.4 

12.4 

7.9 

Credit by exam 

0.7 

2.2 

0.3 

4.4 

2.4 

1.5 

At-risk of not earning sufficient credits 

Regular classroom instruction 

61.0 

72.0 

90.1 

49.0 

68.1 

68.7 

Computer-aided credit accrual 

10.8 

11.0 

5.8 

23.8 

14.7 

31.4 

Trailer course 

3.9 

3.3 

0.0 

7.5 

16.1 

10.8 

Credit by exam 

0.5 

1.9 

0.3 

5.9 

2.3 

2.1 

Did not earn sufficient credits for promotion 

Regular classroom instruction 

43.4 

47.5 

66.3 

48.6 

57.4 

79.3 

Computer-aided credit accrual 

20.8 

20.4 

30.6 

14.2 

16.7 

33.0 

Trailer course 

2.9 

6.7 

0.7 

6.3 

18.2 

2.6 

Credit by exam 

1.8 

3.0 

0.8 

5.4 

2.1 

1.9 

Newly promoted but lacking basic skills 

Regular classroom instruction 

- 

- 

- 

42.1 

62.4 

92.2 

Computer-aided credit accrual 

- 

- 

- 

5.9 

1.9 

5.8 

Trailer course 

- 

- 

- 

1.8 

0.4 

0.0 

Credit by exam 

- 

- 

- 

2.6 

2.6 

0.1 


Source: NGSI activity reports. 


> NGSI instructional methods changed during the course of the grant, with the use of tutoring 
decreasing and the use of counseling and mentoring increasing. 

Table 4.14 includes data for regular and summer terms (except for the 2001-02 school year). During 
spring 2000, more than half of the students participated in tutoring activities (55%), with a significantly 
lower proportion (9%) participating in counseling activities. By spring 2003, only 21% of NGSI 
participants attended tutoring while counseling participation remained high with a participation rate of 
50%. The use of tutorials in summer programs also decreased whereas mentoring increased. 
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Table 4.14. NGSI Instructional Methods (Percent of Students) 


Method 

Regular Terms 

Summer Terms 

Spring 

2000 

2000- 

01 

2001 

-02 

Spring 

2003 

2000 

2001 

2002 

2003 

All NGSI Participants 

Tutoring 

55.4 

76.7 

* 

21.4 

30.6 

30.9 

* 

17.7 

Counseling 

9.5 

60.1 

* 

49.8 

29.6 

27.7 

* 

32.7 

Teaming 

4.2 

41.4 

* 

25.5 

16.3 

10.2 

* 

16.8 

Smaller group 
instruction 

21.2 

49.8 

* 

* 

45.2 

34.0 

* 

* 

Mentoring 

5.5 

39.5 

* 

19.5 

13.6 

17.6 

* 

24.6 

At-risk of not earning sufficient credits 

Tutoring 

57.5 

75.1 

* 

22.6 

33.7 

27.1 

* 

18.6 

Counseling 

7.9 

59.5 

* 

51.1 

23.0 

23.7 

* 

32.6 

Teaming 

3.2 

43.6 

* 

26.8 

13.9 

5.1 

* 

15.9 

Smaller group 
instruction 

22.7 

47.8 

* 

* 

43.3 

29.7 

* 

* 

Mentoring 

5.5 

37.7 

* 

18.3 

7.7 

12.7 

* 

22.8 

Did not earn sufficient credits for promotion 

Tutoring 

46.2 

76.5 

* 

14.6 

28.3 

30.8 

* 

14.4 

Counseling 

6.4 

63.7 

* 

42.3 

47.1 

27.8 

* 

28.6 

Teaming 

7.2 

31.9 

* 

18.5 

11.7 

5.8 

* 

14.0 

Smaller group 
instruction 

24.9 

55.9 

* 

* 

42.9 

26.1 

* 

* 

Mentoring 

5.5 

49.3 

* 

25.0 

19.3 

19.6 

* 

32.1 

Newly promoted but 

lacking basic skills 

Tutoring 

- 

- 

- 

- 

37.6 

38.5 

* 

15.4 

Counseling 

- 

- 

- 

- 

31.7 

39.0 

* 

37.0 

Teaming 

- 

- 

- 

- 

30.6 

20.7 

* 

25.7 

Smaller group 
instruction 

- 

- 

- 

- 

54.1 

48.8 

* 

* 

Mentoring 

- 

- 

- 

- 

23.6 

28.9 

* 

26.4 


Note. Students could be classified as newly promoted only during the summer semesters. 

Comparable activity codes were unavailable for 2001-02 and for some indicators during other terms. 
Source: NGSI activity reports. 
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Successful Program Features 

Districts also reported information on their NGSI programs’ most successful features. Table 4.15 presents 
the most commonly reported successful program features included in NGSI evaluation reports submitted 
by 222 of 226 districts receiving continuation and original funding. 

> Technology-based instruction and iearning emerged as the most consistentiy reported 
successfui program feature. Aithough tutoring was cited as the most successfui feature for 
reguiar terms, it was viewed iess consistentiy as a successfui feature of summer programs. 


Tabie 4.15. Successfui Program Features (Percent of Programs) 



Regular Terms 

Summer Terms 

Feature 

2000-01 

2001-02 

Fall 2002 

2000 

2001 

2002 

Tutoring 

31.50 

25.80 

36.80 

5.9 

15.8© 

4.50 

Technology (self-paced 
programs, NovaNET, PLATO) 

23.2© 

18.2© 

21.8© 

14.4© 

18.00 

8.10 

Technology (programs, software) 

22.4© 

15.2© 

21.8© 

13.50 

9.0© 

4.5© 

Class-size reduction 

10.0 

6.4 

2.3 

12.6© 

7.2 

4.50 

Credit recovery 

13.80 

6.4 

10.3 

24.80 

7.2 

2.3 

Instructional approach 

9.4 

7.2 

11.5© 

14.0© 

12.20 

5.0© 

Teaming students and teachers 

6.2 

4.9 

3.4 

5.4 

3.6 

0.9 

Scheduling (school within school, 
block, flexible) 

7.4 

4.2 

9.2 

2.7 

6.3 

6.3© 

Modified coursework 

9.1 

5.7 

2.3 

10.4 

5.9 

4.50 

Staff-support for program 

11.8© 

4.9 

11.5© 

4.5 

4.5 

3.2 

Staff-additional positions 

5.0 

6.8 

9.2 

0.9 

4.1 

2.3 

Staff-professional development 

6.8 

7.6© 

9.2 

5.9 

8.6 

0.9 

Individual student attention 

10.3 

2.7 

4.6 

6.3 

8.6 

4.50 

Assessment strategies 

8.2 

5.7 

5.7 

4.5 

1.8 

2.3 

Monitoring student progress 

3.8 

6.4 

10.3 

1.4 

2.3 

1.4 

Mentoring program/HOSTS 

7.4 

4.9 

5.7 

3.2 

4.1 

1.4 

Counseling 

5.3 

8.30 

6.9 

5.4 

3.6 

3.2 

Parent support and involvement 

8.5 

7.2 

4.6 

4.5 

5.4 

3.6 

Extended day, Saturday 

2.6 

6.4 

13.80 

1.4 

3.6 

0.0 

Student skill development 

3.8 

1.1 

3.4 

5.0 

2.3 

2.3 

Reading course or strategy 

2.4 

3.0 

6.9 

1.4 

2.7 

0.5 

Individualized academic plan 

1.2 

1.5 

0.0 

0.0 

2.3 

0.9 

Summer program 

0.3 

0.0 

0.0 


12.6© 

3.2 

Provision of transportation 

0.5 

0.0 

1.8 



0.0 

Incentives or rewards 

1.2 

2.3 

4.6 


2.3 

0.5 


Note. Numeric symbols (e.g., O) indicate the most frequently identified features by rank order. 
Source: NGSI program reports. 


The most frequently identified successful program features are described below. 

Tutoring. Districts identified tutoring as one of the most successful NGSI program features during the 
regular terms. Respondents noted the effectiveness of tutorials — ^held before, during, and after school or 
on Saturdays — in helping students complete coursework, understand concepts, make-up work, do 
homework, and get one-on-one, individualized attention. Tutorials, according to reports, enhanced student 
achievement, improved grades, allowed credit recovery and accrual, and prevented credit losses. 
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Technology (self-paced). Districts identified the implementation of self-paced instructional 
technology as one of the top two most successful features of their programs. Respondents most often 
reported the implementation of PLATO and NovaNET software in lahs for tutorials, credit recovery, 
or credit accrual. According to many, self-paced programs benefited students hy diagnosing needs, 
individualizing instruction, meeting various learning styles, allowing a flexible pace, supporting the 
acquisition of credits to stay on grade level, and providing an alternative method to deliver instruction 
for students who had not been successful in traditional settings. 

Technology (hardware, software, and programs). In addition to self-paced instructional 
programs, districts cited positive effects associated with the acquisition of educational technology 
hardware, software, and programs to support student learning. Districts noted beneficial features such 
as updated computers and computer labs, subject-specific labs (algebra, math, English), individual 
laptops, graphing calculators, educational programs (e.g.. Accelerated Reader, Accelerated Math, 
River Deep, Computer Curriculum Corporation [CCC]), and computer applications (word processing, 
graphics). Programs reported positive impacts of technology on the individualization of instruction 
and improved content coverage. Technology also enhanced students’ communication and technical 
skills, motivation to learn, and academic achievement. 

Instructional approach. Although not widely cited during regular terms, districts identified the 
instructional approach as a “top five” most successful feature during summer terms. Districts 
described smaller classes; active hands-on activities allowing students to understand abstract 
concepts; small group, partner, and collaborative learning; student self-selection of reading materials 
or learning activities; mastery learning; and individualized instruction. In general, reduced student-to- 
teacher ratios in summer terms supported more learner-centered instructional approaches. 

Class-size reduction, credit recovery, staff support for NGSI program, and individual 
student attention. Other successful NGSI program features that districts designated less 
consistently across terms include reduced class size allowing for more personalized instruction, 
opportunities for credit recovery, staff commitment and support for the NGSI program, and 
individualized student attention. 


STUDENT OUTCOMES 

This section presents findings on the effect of grant resources on targeted students. To the extent possible, 
researchers also examine associations between NGSI program elements and student outcomes. We first 
present results for student core-content course passing rates. Next, to estimate NGSI program impact, 
comparisons are made between NGSI and non-NGSI students on outcomes, including school attendance, 
performance on state-level assessments, and retention. Einally, the relationships between NGSI student- 
level and district-level variables and student academic achievement and grade-level retention are explored 
using complex statistical models. 

Passing Rates for Core-Content Courses 

NGSI programs reported on credit accrual and retrieval in courses typically taken during the 
ninth-grade year, such as English I, Algebra I, World Geography, Biology, and Integrated 
Physics and Chemistry (IPC). In addition, districts provided credit information on other courses 
offered to targeted ninth graders, such as Algebra II, Geometry, World History, U.S. History, and 
English II. The percentages of NGSI students passing courses are reported in Eigures 4.2 and 4.3 
Passing rates are disaggregated by student eligibility categories in Tables 4.16 through 4.20. 
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> Course passing rates were relatively stable across NGSI grant terms, with about 70% of NGSI 
students passing Algebra I during regular terms and about three-fourths or more of students 
passing Biology, IPC, World Geography, and English I. 

As illustrated in Figure 4.2, NGSI students had lower passing rates for Algebra I compared to other core 
subject areas, with 72% or less students passing algebra during regular terms. Course passing rates for 
other core courses were comparable, with three-quarters or more of students passing required courses. 



□ 2000-01 02001-02 02002-03 


Figure 4.2. Percentage of NGSI participants passing core courses during regular terms. 

Source: NGSI activity reports. 

> The vast majority of NGSI students who participated in summer terms passed core courses 
(about 80% to 95%); however, the number of students enrolled during the summer was far less 
than for regular terms. 

Figure 4.3 shows that NGSI students who participated during summer terms had notably higher passing 
rates for all core courses and passing rates were generally on an upward trend until summer 2003. 



□ 2000 D2001 02002 02003 

Figure 4.3. Percentage of NGSI participants passing core courses during summer terms. 

Source: NGSI activity reports. 
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Mathematics 

> Students in ninth grade for the first time had higher passing rates for mathematics courses 
than students who did not earn sufficient credits for promotion. 

> During the summer terms, course- passing rates for mathematics were the higher across aii 
eiigibiiity categories but few students participated. 

Students who did not earn enough credits for promotion had the lowest mathematics passing rates. In 
general, lower numbers of ninth graders enrolled in mathematics courses during summer terms, and 
participating students had passing rates 7 to 20 percentage points higher than ninth graders enrolled in 
regular terms (Table 4.16). 


Tabie 4.16. Percentage of Students Passing Mathematics Courses 


Term 

Algebra I 

Algebra II 

Geometry 

N 

Pass 

N 

Pass 

N 

Pass 

All NGSI participants 

2000-01 

40,391 

71.5 

545 

89.2 

6,662 

82.3 

2001-02 

36,957 

69.3 

328 

78.4 

6,328 

80.1 

2002-03 

42,313 

68.9 

787 

82.2 

6,874 

74.3 

Summer 2000 

3,407 

81.0 

110 

86.5 

163 

88.0 

Summer 2001 

4,494 

85.2 

102 

90.4 

361 

88.0 

Summer 2002 

3,853 

88.3 

53 

96.4 

294 

92.5 

Summer 2003 

4273 

87.3 

100 

83.0 

393 

88.3 

At-risk of not earning sufficient credits 

2000-01 

31,833 

73.3 

468 

90.8 

4,927 

86.3 

2001-02 

31,189 

70.9 

260 

83.1 

4,842 

83.9 

2002-03 

35,585 

70.5 

655 

85.0 

4,958 

81.3 

Summer 2000 

1,428 

85.6 

81 

87.8 

131 

89.4 

Summer 2001 

3,434 

85.9 

56 

89.3 

220 

90.5 

Summer 2002 

3,041 

89.5 

45 

97.8 

151 

93.2 

Summer 2003 

3,602 

87.7 

78 

84.6 

220 

85.9 

Did not earn sufficient credits for promotion 

2000-01 

5,402 

62.5 

68 

77.9 

1,552 

68.9 

2001-02 

2,613 

58.2 

40 

47.5 

922 

59.3 

2002-03 

4,273 

55.2 

75 

56.0 

829 

52.4 

Summer 2000 

672 

69.3 

2 

* 

11 

91.7 

Summer 2001 

457 

78.8 

38 

97.4 

83 

78.9 

Summer 2002 

590 

84.0 

8 

* 

137 

91.4 

Summer 2003 

406 

83.7 

14 

35.7 

104 

88.5 


Note. Passing percentage based on number of courses taken. Columns may not sum 
because some students lack eligibility data. 

*Passing percentages not reported for groups with less than ten students. 

Source: NGSI activity reports. 
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Science 

> About three-fourths of NGSI students passed Biology and IPC during regular terms. 

> Course participation rates were lower for summer terms, but course-passing rates were higher 
across all eligibility categories. 

Substantial numbers of NGSI students enrolled in either Biology or IPC courses during regular terms. 
Students not earning sufficient credits for promotion had lower passing rates than students at risk of not 
earning sufficient credits. Enrollment levels declined during summer terms, and students’ science passing 
rates were 12 to 38 percentage points higher. In addition, the disparity between students at-risk of not 
earning credits and those repeating coursework was not apparent during summer terms (Table 4.17). 


Table 4.17. Percentage of Students Passing Science Courses 


Term 

Biology 

IPC 

N 

Pass 

N 

Pass 

All NGSI participants 

2000-01 

27,240 

80.4 

22,471 

78.7 

2001-02 

22,772 

75.1 

15,027 

75.2 

2002-03 

24,016 

73.4 

23,408 

74.4 

Summer 2000 

532 

87.5 

650 

87.0 

Summer 2001 

1,326 

91.5 

887 

92.2 

Summer 2002 

705 

95.1 

1,061 

93.2 

Summer 2003 

1,504 

89.4 

1,343 

88.9 

At-risk of not earning sufficient credits 

2000-01 

21,870 

82.5 

17,286 

80.8 

2001-02 

19,084 

76.9 

12,440 

77.0 

2002-03 

20,134 

76.0 

19,064 

77.8 

Summer 2000 

244 

90.8 

195 

90.3 

Summer 2001 

894 

93.1 

591 

92.6 

Summer 2002 

537 

95.2 

817 

92.7 

Summer 2003 

1,238 

88.9 

1,135 

88.3 

Did not earn sufficient credits for promotion 

2000-01 

4,111 

67.6 

3,725 

70.7 

2001-02 

1,946 

60.5 

1,308 

62.2 

2002-03 

2,762 

56.9 

2,846 

55.0 

Summer 2000 

158 

77.7 

204 

85.2 

Summer 2001 

205 

89.8 

132 

93.7 

Summer 2002 

131 

96.6 

187 

93.2 

Summer 2003 

192 

91.1 

154 

92.9 


Note. Passing percentage based on number of courses taken. Columns 
may not sum because some students lack eligibility data. IPC=Integrated 
Physics and Chemistry. Source: NGSI program reports. 
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Social Studies 

> Students participating in NGSi programs enroiied in Worid Geography courses more often 
than other sociai studies ciasses, with the majority of students passing (64% to 93%). 

> Most NGSi students who compieted sociai studies coursework during summer terms passed 
(80% to 98%). 

Regular term passing rates for the three social studies courses were considerably lower than summer term 
passing rates. Also during regular terms, ninth graders not earning sufficient credits for promotion 
generally had lower passing rates for social studies classes compared to other student groups. 


Tabie 4.18. Percentage of Students Passing Sociai Studies Courses 


Term 

World Geography 

World History 

U.S. History 

N 

Pass 

N 

Pass 

N 

Pass 

All NGSI participants 

2000-01 

36,206 

80.3 

4,985 

73.6 

4,394 

85.8 

2001-02 

27,389 

76.6 

4,729 

75.0 

869 

70.4 

2002-03 

40,092 

78.2 

5,725 

66.3 

583 

68.8 

Summer 2000 

942 

88.2 

146 

81.6 

283 

82.1 

Summer 2001 

1,597 

92.1 

284 

90.0 

569 

88.6 

Summer 2002 

1,634 

93.2 

231 

90.7 

62 

97.1 

Summer 2003 

2213 

91.9 

274 

90.5 

57 

98.2 

At-risk of not earning sufficient credits 

2000-01 

30,331 

81.0 

2,166 

80.2 

3,015 

88.4 

2001-02 

23,639 

77.3 

3,442 

80.9 

430 

71.4 

2002-03 

34,446 

78.9 

2,654 

74.8 

273 

78.8 

Summer 2000 

344 

91.8 

87 

86.8 

116 

86.7 

Summer 2001 

1,125 

91.9 

171 

88.6 

439 

88.1 

Summer 2002 

1,251 

93.2 

124 

90.6 

40 

90.9 

Summer 2003 

1,863 

91.7 

134 

91.0 

6 

* 

Did not earn sufficient credits for promotion 

2000-01 

4,087 

74.1 

2,288 

66.4 

1,141 

79.0 

2001-02 

1,617 

66.9 

1,048 

56.5 

355 

71.8 

2002-03 

2,671 

64.1 

2,761 

57.0 

247 

56.3 

Summer 2000 

244 

83.2 

30 

66.7 

120 

70.2 

Summer 2001 

179 

91.7 

84 

93.6 

52 

88.7 

Summer 2002 

247 

92.8 

92 

90.4 

21 

100.0 

Summer 2003 

188 

91.5 

105 

86.7 

17 

94.1 


Note. Passing percentage based on number of courses taken. Columns may not sum 
because some students lack eligibility data. . 

* Passing percentages not reported for groups with less than ten students. 
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English Language Arts 

> The majority of A/GS/ students enrolled in English I passed, with passing rates ranging from 
77% to 94%. 

> Students in ninth grade for the first time had higher passing rates than students who did not 
earn sufficient credits for promotion. Course participation rates were lower in the summer, but 
course-passing rates were higher across all eligibility categories. 

Although students’ not earning sufficient credits for promotion had the lowest passing rates during the 
regular terms, rates were more equal across groups during summer terms. However, significantly lower 
numbers of ninth graders enrolled in English courses during summer terms, and participating students had 
passing rates 10 to 39 percentage points higher than ninth graders enrolled in regular terms (Table 4.19). 


Table 4.19. Percentage of Students Passing 
English Language Arts Courses 


NGSI Semester 

English I 

English II 

N 

Pass 

N 

Pass 

All NGSI participants 

2000-01 

38,927 

80.7 

3,738 

70.4 

2001-02 

34,149 

76.5 

3,103 

65.5 

2002-03 

39,801 

77.2 

3,949 

61.7 

Summer 2000 

2,572 

87.4 

146 

82.1 

Summer 2001 

3,703 

91.3 

255 

89.7 

Summer 2002 

3,164 

93.7 

267 

91.7 

Summer 2003 

3,753 

91.5 

259 

91.9 

At-risk of not earning sufficient credits 

2000-01 

32,142 

81.3 

1,149 

75.4 

2001-02 

29,106 

77.5 

1,543 

69.9 

2002-03 

34,856 

78.6 

1,398 

72.5 

Summer 2000 

978 

93.3 

64 

92.8 

Summer 2001 

2,882 

91.7 

46 

79.2 

Summer 2002 

2,456 

94.3 

84 

98.9 

Summer 2003 

3,215 

91.2 

61 

86.9 

Did not earn sufficient credits for promotion 

2000-01 

4,287 

74.2 

2,418 

67.5 

2001-02 

2,138 

68.7 

1,216 

60.9 

2002-03 

3,696 

65.9 

2,316 

55.0 

Summer 2000 

767 

74.9 

29 

65.6 

Summer 2001 

415 

87.5 

133 

89.0 

Summer 2002 

558 

91.7 

175 

88.7 

Summer 2003 

323 

92.6 

155 

94.8 


Note. Passing percentage based on number of courses taken. Columns 
may not sum because some students lack eligibility data. 
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Course Passing Rates for Newly Promoted Ninth Graders 

> Newly promoted ninth grade students who participated in summer coursework had high 
passing rates (89% to 98%); however, only a few students participated. 

Very few newly promoted ninth graders participated in summer course work for credit. Passing rates for 
participating students, however, were generally high, with the highest passing rates achieved in English I 


Table 4.20. Percentage of Newly Promoted Ninth Graders Passing Courses 


Term 

Algebra I 

Biology 

Word 

Geography 

English I 

N 

Pass 

N 

Pass 

N 

Pass 

N 

Pass 

Summer 2000 

214 

88.5 

28 

100.0 

118 

91.5 

101 

98.1 

Summer 2001 

104 

97.1 

62 

93.6 

45 

94.7 

87 

97.7 

Summer 2002 

56 

88.5 

6 

* 

53 

98.2 

39 

97.7 

Summer 2003 

10 

* 

2 

* 

39 

97.4 

20 

90.0 


Note. Passing percentage based on number of courses taken. Columns may not sum because some 
students lack eligibility data. *Passing percentages not reported for groups with ten students or fewer. 
Source: NGSI activity reports. 


Comparisons Between NGSI and Non-NGSI Students 

Estimating the impact of a program intervention requires the measurement of student outcomes after 
program implementation and comparisons to outcomes students would have achieved in the absence of 
the intervention. Thus, a comparahle group of non-served students is needed. The NGSI student-level 
database contained records for all ninth graders in each NGSI-supported district, both NGSI and non- 
NGSI served students. It is impossible to form a true comparison group because NGSI-served students 
(by grant mandate) included those repeating ninth grade or at risk of being retained; thus, students who 
did not receive services were less at risk. Although non-served students might include some at-risk 
students, the majority of students in the comparison group have fewer risk factors. With these limitations 
in mind, an imperfect but more comparable group of non-served students was created by restricting the 
comparison group to those students who were similar to the NGSI-served students on six dimensions: 
ethnicity, gender, economic disadvantage status (based on eligibility for free or reduced price lunch), 
limited English proficiency, whether or not they repeated ninth grade, and district attended. 

The characteristics of NGSI and comparison students are given in Table 4.21. Of 28,366 students in 
cohort 1, half (14,183) are NGSI students and half are non-NGSI students. Groups have the same 
distribution for the matching variables. Cohort 2 includes a substantially greater number of students 
because more students were served in NGSI during this program year; however, student demographic 
characteristics are similar to cohort 1 except for a lower proportion of male students and a slightly greater 
proportion of students repeating ninth grade. The number of students decreases slightly for cohort 3 (to 
56,532). The demographic characteristics are similar to the previous two cohorts, but the percentage of 
repeating ninth graders increases again. 
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Table 4.21. Characteristics of Comparison Groups for NGSI Outcome Analysis 



Cohort 1 (1999-00) 

Cohort 2 (2000-01) 

Cohort 3 (2001-02) 

Cohort 4 (2002-03) 

NGSI 

n=14,183 

Non- 

NGSI 

n=14,183 

NGSI 

n=31,272 

Non- 

NGSI 

n=31,272 

NGSI 

n=28,266 

Non- 

NGSI 

n=28,266 

NGSI 

n=28,061 

Non- 

NGSI 

n=28,061 

Gender 







Male 

57.2 

57.2 

51.8 

51.8 

53.9 

53.9 

53.8 

53.8 

Female 

42.8 

42.8 

48.2 

48.2 

46.1 

46.1 

46.2 

46.2 

Race/Ethnicity 







Hispanic 

51.7 

51.7 

51.3 

51.3 

51.5 

51.5 

49.6 

49.6 

African American 

19.5 

19.5 

17.8 

17.8 

18.3 

18.3 

16.7 

16.7 

White 

27.4 

27.4 

29.0 

29.0 

28.7 

28.7 

31.6 

31.6 

Other/NA 

1.4 

1.4 

1.8 

1.8 

1.5 

1.5 

2.0 

2.0 

Repeating 9th grade 

9.4 

9.4 

11.4 

11.4 

12.5 

12.5 

9.3 

9.3 

Limited English 
proficient 

10.2 

10.2 

8.6 

8.6 

8.9 

8.9 

7.0 

7.0 

Economically 

disadvantaged 

53.6 

53.6 

51.8 

51.8 

54.5 

54.5 

50.8 

50.8 


Note. Cohort 1, N=28,366; Cohort 2, N=62,544; Cohort 3, N=56,532. Cohort 4, N=56,122. 


Attendance Rates 

Time-series analysis is employed to evaluate the NGSI program impact on student attendance rates. Five 
years of attendance data are available for cohort 1 students (1997 through 2002), five years for cohort 2 
(1998 to 2003), four years for cohort 3 (1999 to 2003), and three years for cohort 4 (2001 to 2003). The 
analysis is restricted to matched students with valid data for all years. The number of students included in 
the analysis is given in Table 4.22. Of the 25,351 students in cohort 1, 23,577 are in the ninth grade for 
the first time and 1,774 are repeating ninth grade. Cohorts 2 and 3 include a greater number of students 
and an increasingly larger proportion of students repeating ninth grade. The proportion of students 
repeating ninth grade decreased for cohort 4. 


Table 4.22. Number of Students Included In Analysis 



NGSI 

Non-NGSI 

Total 

Cohort 1 
(1999-00) 

All Students 

12,728 

12,623 

25,351 

Eirst Time 

11,801 

11,776 

23,577 

Repeating 

927 

847 

1,774 

Cohort 2 
(2000-01) 

All Students 

30,370 

30,225 

60,595 

Eirst Time 

26,944 

26,832 

53,776 

Repeating 

3,426 

3,393 

6,819 

Cohort 3 
(2001-02) 

All Students 

27,516 

27,368 

54,884 

First Time 

24,106 

23,996 

48,102 

Repeating 

3,410 

3,372 

6,782 

Cohort 4 
(2002-03) 

All Students 

26,311 

26,068 

52,379 

First Time 

23,902 

23,676 

47,578 

Repeating 

2,409 

2,392 

4,801 
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Attendance Rates for All NGSI Students 

> In general, NGSI students’ attendance rates have not improved. 

Longitudinal attendance rate data for students in each of the four cohorts (see Table 4.23) show that 
compared to non-NGSI students, NGSI students have lower attendance rates for all years, although the 
general downward trend in attendance rates is similar for the two groups. Still, the gap between 
comparison groups grows wider each year. For example, for cohort 1 there is a 0.5 percentage-point 
gap in 1997-98, 0.6 in 1998-99, and 0.8 in 1999-00. Net changes over time for cohorts 1 and 2 are 
calculated as the difference in attendance rates two years prior to NGSI participation and two years 
after participation. For both cohorts, NGSI students had a greater decline in attendance rates than non- 
NGSI students (a 5.7 percentage point decline for cohort 1 and a 5.2 point decline for cohort 2). 
Although attendance data for fewer years are available for cohorts 3 and 4, trends are similar to other 
student cohorts. In general, participation in the NGSI program did not alter the downward pattern in 
high school students’ attendance trends. 


Table 4.23. Longitudinal Attendance Rates for All Students. 



Cohort 1 

Cohort 2 

Cohort 3 

Cohort 4 


1999-00 

2000-01 

2001-02 

2002-03 


NGSI 

Non-NGSI 

NGSI 

Non-NGSI 

NGSI 

Non-NGSI 

NGSI 

Non-NGSI 


Students 

Students 

Students 

Students 

Students 

Students 

Students 

Students 

1997-1998 

95.5 

96.0 







1998-1999 

94.8 

95.4 

95.5 

96.0 





1999-2000 

94.1 

94.9 

95.1 

95.7 

95.5 

95.8 



2000-2001 

92.4 

93.6 

94.1 

95.0 

94.5 

95.0 

95.3 

95.7 

2001-2002 

89.8 

91.7 

92.9 

94.1 

93.5 

94.2 

94.6 

95.0 

2002-2003 



90.3 

92.4 

91.7 

92.9 

93.6 

94.2 

Change 

-5.7 

-4.3 

-5.2 

-3.6 

-3.8 

-2.9 

-1.7 

■ 

In 


Note. Shaded cells denote NGSI participation year. 


Attendance Rates for First-Time and Repeat Ninth Graders 

Student attendance rates, disaggregated by students in a grade for the first time and those who are 
repeating a grade level, are displayed in Figures 4.4 through 4.7 for the four student cohorts. 

> First-time ninth graders have substantially higher attendance rates than repeat ninth graders 
for both NGSI and non-NGSI comparison groups. NGSI first-time ninth graders, however, have 
lower attendance rates than their non-NGSI peers. 

Results in Figure 4.4 compare attendance trends for first-time and repeat ninth graders in cohort 1 . The 
two uppermost graph lines represent first-time ninth graders and the two lower graph lines represent 
repeat ninth graders. The data for first-time ninth graders clearly shows, not only a decrease in attendance 
rates across time, but also a widening gap between NGSI and non-NGSI students. One expected outcome 
of the NGSI program is a positive change in attendance rates, which would be suggested if the gap 
between the two groups stayed the same or declined; however, this does not occur. 
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98 . 0 % 

96 . 0 %- 
94 . 0 % 

92 . 0 % 

90 . 0 % 

88 . 0 % 

86 . 0 %- 
84 . 0 % 

82 . 0 %- 
80 . 0 % ■ 

1997-1998 1998-1999 1999-2000 Cohort 1 2000-2001 2001-2002 

Figure 4.4. Attendance rates for cohort 1(1999-00), first-time and repeat ninth graders. 

> Attendance rates for repeat NGSi ninth graders (in cohort 1) surpassed non-NGSi comparison 
groups. Aithough attendance rates for repeat NGSi students deciined across time, this change 
was reduced siightiy during the NGSi program impiementation year. 

Attendance rate trends for repeat ninth graders differ from those for first-time ninth graders. First, NGSI 
students have higher attendance rates across all years. Repeat NGSI and non-NGSI students have 
essentially the same attendance rates during the 1997-98 school year (93.1% and 92.8% respectively), and 
both groups experience a decline in attendance rates across time. However, unlike the gradual decline in 
attendance rates for non-NGSI students, NGSI students experienced a change in their long-term trend the 
year of the NGSI program. Looking at the graph line in Figure 4.4, the attendance rate decline is 
comparatively less steep for the NGSI program year. However, the change is not permanent and the 
following year (2000-2001), NGSI student attendance rates resumed their same downward trend, and hy 
the last year data are available (2001-2002), NGSI students repeating ninth grade have comparable 
attendance rates to non-NGSI served students. 

> NGSI student attendance patterns for first-time and repeat ninth graders remained relatively 
consistent across program years. 

Students in cohort 2 exhibit the same patterns as seen for cohort 1 (Figure 4.5). First-time NGSI ninth 
graders also have lower attendance rates across time than non-NGSI students. However, the gap is 
slightly smaller and does not grow as large across time. Repeating NGSI ninth graders have higher 
attendance rates than non-NGSI students and the difference in attendance rate trends associated with 
NGSI participation is also seen for cohort 2 students, but to a much smaller extent. 
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Figure 4.5. Attendance rates for cohort 2 (2000-01), first-time and repeat ninth graders. 


First-time ninth graders in cohorts 3 and 4 have similar attendance patterns to cohorts 1 and 2. Flowever, 
data are unavailable to track student attendance across time (see Figures 4.6 and 4.7). 
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1 NGSI, first 

-■-Non-NGSI, first I 

1 -0- NGSI, repeat 

- 0 - Non-NGSI, repeat | 



2000-2001 2001-2002 2002-2003 (Cohort 4) 

Figure 4.7. Attendance rates for cohort 4 (2002-03), first-time and repeat ninth graders. 


State-Level Assessments 

The NGSI was expected to increase student performance on state-level assessments. To evaluate the 
NGSI effect on student academic performance TAAS reading and mathematics scores, and TAKS English 
language arts and mathematics scores were merged with the student-level database. TAAS performance is 
measured hy whether or not a student passed the math and reading suhtests. Available scores include 8th 
grade TAAS, the year prior to NGSI participation, and 10th grade exit-level TAAS, the year after NGSI 
participation. The sample for cohorts 1 and 2 is restricted to students who were promoted to tenth grade 
and have either TAAS for both testing years.' The introduction of a new state assessment, TAKS, 
during the 2002-03 school year resulted in its use as an outcome measure for students in cohorts 
3 and 4. Because TAKS and TAAS are not equivalent tests, the 8th grade TAAS scores are not 
shown for these two cohorts. The 10th grade TAKS assessment passing rates are given for cohort 
3 students. Because the TAKS assessment, for the fist time, was administered to 9th graders in 
2002-03, it afforded researchers the ability to use an outcome measured the same year as 
program participation for cohort 4 students. 


First-Time Ninth Graders 

> NGSI students had lower TAAS passing rates for both reading and math compared to non- 
NGSI students, but the achievement gap between groups narrowed In reading. Despite 
encouraging results for TAAS, the achievement gap widened substantially for students In 
cohorts 3 and 4 who completed the TAKS. 

Of cohort 1 NGSI students in ninth grade for the first time, 83.2% passed TAAS reading as eighth 
graders, while 92.4% of non-NGSI students passed (an achievement gap of 9.2 percentage points). Two 
years later, 84.0% of NGSI students passed TAAS reading in the tenth grade, compared to 92.8% of non- 
NGSI students (an achievement gap of 8.8 percentage points). Although non-NGSI students outperformed 
NGSI students both years, the performance gap narrowed slightly (0.4 percentage points) (Table 4.24). 


* Absent, ARD exempt, and LEP exempt student scores are not used. 


99 


For cohort 2, non-NGSI students also passed TAAS at a greater rate than NGSI students, however the 
achievement gap between the two groups continued to decrease. By the tenth grade, non-NGSI students 
were passing TAAS reading at a rate only 3.7 percentage points higher than NGSI students, which is half 
the eighth grade gap. The analysis is repeated for TAAS math. Large achievement gaps between NGSI 
and non-NGSI students also exist for cohort 1 and 2 students, with the achievement gap narrowing 
slightly more for cohort 2. 

Despite encouraging results for TAAS, outcomes for cohort 3 students as measured by the new state 
assessment (Texas Assessment of Knowledge and Skills — TAKS) are less promising. Only two-thirds of 
NGSI students passed the TAKS lOth-grade reading (65%) and mathematics (63%) assessments. 
Moreover, the achievement gap between NGSI and non-NGSI students widened substantially, with a 
large achievement gap in reading (15.6 percentage points) and math (19.8 percentage points). Results for 
cohort 4 students who completed the 9th grade TAKS, however, show a smaller achievement gap in 
reading (7.8 percentage points), but the achievement gap in math remained high (17.7 points). 


Table 4.24. TAAS and TAKS Passing Rates for First-Time Ninth Graders 



Reading 

Math 

NGSI 

Students 

Non-NGSI 

Students 

Achievement 

Gap 

NGSI 

Students 

Non-NGSI 

Students 

Achievement 

Gap 

Cohort 1 

(1999-00) 

n=14,895 

TAAS 8th 

83.2 

92.4 

9.2 

80.1 

91.3 

11.2 

TAAS 10th 

84.0 

92.8 

00 

00 

81.8 

92.6 

10.8 

Cohort 2 
(2000-01) 
n=33,728 

TAAS 8th 

87.1 

94.6 

7.5 

87.9 

95.8 

7.9 

TAAS 10th 

93.9 

97.6 

3.7 

89.7 

96.3 

6.6 

Cohort 3 
(2001-02) 
n=29,775 

TAKS 10th 

64.6 

80.2 

15.6 

62.9 

82.7 

19.8 

Cohort 4 

(2002-03) 

n=33,690 

TAKS 9th 

84.5 

92.3 

7.8 

62.7 

80.4 

17.7 


Note. Students in cohort 3 completed the tenth-grade TAKS assessment in 2002-03 (the first administration 
year for the new assessment). Cohort 4 students completed the ninth-grade TAKS during the 2002-03 school 
year. 


Repeat Ninth Graders 

> NGSI repeat ninth graders have similar TAAS and TAKS passing rates compared to non-NGSI 
students for both reading and mathematics, however, for both student groups, passing rates 
declined substantially for the new TAKS assessment. 

For repeat ninth graders, eighth-grade TAAS scores are from two years prior to NGSI participation. The 
percent of students passing TAAS for each group is given in Table 4.25. For cohort 1, the TAAS reading 
passing rates for NGSI and non-NGSI students were very similar. For the eighth grade TAAS reading, 
55.2% of NGSI-served students and 56.3% of non-NGSI students passed, resulting in an achievement gap 
of only 1.1 percentage points. Three years later, this gap had increased slightly to 1.7 percentage points. 
The increasing achievement gap across time also is apparent for TAAS math passing rates. 
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Table 4.25. TAAS and TAKS Passing Rates for Repeat Ninth Graders 



Reading 

Math 

NGSI 

Students 

Non-NGSI 

Students 

Achievement 

Gap 

NGSI 

Students 

Non-NGSI 

Students 

Achievement 

Gap 

Cohort 1 

(1999-00) 

n=666 

TAAS 8th 

55.2 

56.3 

1.1 

46.9 

50.5 

3.6 

TAAS 10th 

70.5 

72.2 

1.7 

68.8 

74.1 

5.3 

Cohort 2 
(2000-01) 
n= 1,495 

TAAS 8th 

68.1 

72.7 

4.6 

64.0 

66.2 

2.2 

TAAS 10th 

88.7 

87.4 

-1.3 

79.2 

77.3 

-1.9 

Cohort 3 
(2001-02) 
n=365 

TAKS 10th 

49.4 

57.8 

8.4 

39.3 

42.2 

2.9 

Cohort 4 
(2001-02) 
n= 1,643 

TAKS 9th 

70.3 

72.1 

1.8 

34.8 

38.8 

4.0 


NGSI students in cohort 2 have a lower TAAS passing rate (4.6 percentage points) on eighth-grade 
reading than non-NGSI students. Three years later, however, these same NGSI students passed the tenth- 
grade TAAS at a slightly higher rate than non-NGSI students, eliminating the achievement gap. The same 
pattern is seen for TAAS mathematics. The 2.2 percentage point achievement gap for eighth-grade TAAS 
passing rates is eliminated hy tenth grade, when NGSI students pass TAAS math at a slightly higher rate 
than non-NGSI students. 

For both student groups, passing rates declined substantially for the new TAKS assessment compared to 
results for the TAAS. Across cohorts 3 and 4, passing rates for NGSI and non-NGSI students are similar 
for both reading and math, except the tenth-grade TAKS reading passing rate for NGSI students in cohort 
3 was 8.4 percentage points lower than non-NGSI students. For cohort 4 students, the difference was only 
1.8 percentage points. 

Retention Rates 

> Although NGSI student retention rates remain high, evidence for four program years reveals 
that NGSI retention rates have decreased more than rates for non-NGSI students. 

The primary goal of the NGSI program is to reduce the number of students retained in the ninth grade. To 
determine if this occurs, we examine retention rates for all students in the four cohorts, including first- 
time ninth graders and repeat ninth graders. As Table 4.26 shows, 29.5% of the 1999-00 NGSI students 
were retained in ninth grade, compared to 18.3% for non-NGSI students. Students in the following three 
years had progressively lower retention rates. First-time NGSI ninth graders in 1999-00 had a 29.0% 
retention rate, compared to 16.7% for non-NGSI served students. Students repeating ninth grade had a 
much higher retention rate, for both NGSI and non-NGSI students, and the rates were comparable across 
comparison groups. 

Across all years and student groups, NGSI students had higher retention rates than non-NGSI students, 
except for repeating ninth-graders in 2001-02 and 2002-03, who actually had a slightly lower retention 
rate than the comparison group (19.7% compared to 23.4% in 2001-02). Across time, NGSI students 
experienced greater declines in their retention rates, for all groups. Examining the difference in retention 
rates from 1999-00 to 2002-03, NGSI students decreased their retention rate 7.7 percentage points, 
compared to a 3.3 point reduction for non-NGSI students. First-time and repeat ninth graders experienced 
similar declines, although the difference was smaller for the repeat ninth graders because their retention 
rate increased in 2002-03. 
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Table 4.26. Retention Rates of Ninth Grade Students 



NGSI Students 

Non-NGSI Students 

Number of 
Students 

Percent 

Retained 

Number of 
Students 

Percent 

Retained 

All students 

1999-00 

14,183 

29.5 

14,183 

18.3 

2000-01 

31,272 

24.5 

31,272 

15.6 

2001-02 

28,266 

23.3 

28,266 


2002-03 

28,061 

21.8 

28,061 


First-time 

Ninth 

1999-00 

12,848 

29.0 

12,848 

16.7 

2000-01 

27,714 

23.6 

27,714 

13.7 

2001-02 

24,725 

23.8 

24,725 

13.8 

2002-03 

25,461 

21.4 

25,461 

13.6 

Repeating 

Ninth 

1999-00 

1,335 

33.5 

1,335 

33.7 

2000-01 

3,558 

31.3 

3,558 


2001-02 

3,541 

19.7 

3,541 

23.4 

2002-03 

2,600 

26.6 


28.7 

Reduction 
in retention^ 

All students 

— 

-7.7 

— 

-3.3 

First-time 9th 

— 

-7.6 

— 

-3.1 

Repeating 9th 

- 

-6.9 

- 

-5.0 


“Difference between 1999-00 and 2002-03. 


Retention by Socioeconomic Status 

> First-time ninth graders had greater declines in retention rates than non-NGSI students; the 
retention rate declines were similar for economically disadvantaged and advantaged students. 

Table 4.27 presents retention rates for first-time ninth graders by economic disadvantage status. Across 
all program years, economically disadvantaged NGSI students had a higher retention rate than non- 
economically disadvantaged students. From 1999-00 to 2002-03, the retention rate for economically 
disadvantaged NGSI students declined from 34.0% to 26.9% - a reduction of 7.1 percentage points. 
During this same time period, the retention rate for non-NGSI students who were economically 
disadvantaged declined 2.7 percentage points, from 22.7% to 20.0%. Non-economically disadvantaged 
NGSI students in 2002-03 experienced an 11.0 percentage point decline in retention rates from students in 
1999-00, a decrease is larger than the decline experienced by non-NGSI students and for economically 
disadvantaged NGSI students. 


102 

























Table 4.27. Retention Rates for First-Time Ninth Graders, by Economic Indicators 



NGSI Students 

Non-NGSI Students 

Number of 
Students 

Percent 

Retained 

Number of 
Students 

Percent 

Retained 

Economically 

disadvantaged 

1999-00 

6,771 

34.0 

6,771 

22.7 

2000-01 

14,108 

28.1 

14,108 

18.7 

2001-02 

13,227 

27.8 

13,227 

19.2 

2002-03 

12,730 

26.9 

12,730 

20.0 

Non-economically 

disadvantaged 

1999-00 

6,077 

23.5 

6,077 

10.0 

2000-01 

13,606 

19.0 

13,606 

8.5 

2001-02 

11,498 

19.3 

11,498 

7.6 

2002-03 

12,731 

15.9 

12,731 

7.3 

Reduction in 
retention“ 

Economically 

disadvantaged 

- 

-7.1 

- 

-2.7 

Non-economically 

disadvantaged 

- 

-7.6 

- 

-2.7 


“Difference between 1999-00 and 2002-03. 


> Retention rates for both economically advantaged and disadvantaged NGSI students who 
were repeating ninth grade decreased across time; however, NGSI non-economically 
disadvantaged students had a larger reduction compared to their non-NGSI peers. 

Table 4.28 presents retention rates for repeat ninth graders by economic disadvantage status. Generally, 
NGSI students had lower retention rates than non-NGSI students. While retention rates declined for both 
groups across years, there was a greater decline for non-economically disadvantaged NGSI students. 
Across four program years, economically disadvantaged NGSI and non-NGSI students experienced 
similar reductions in retention rates (6.2 and 6.8 percentage points, respectively). On the contrary, non- 
economically disadvantaged NGSI students reduced their retention rates by 8.1 percentage points, a 
substantial difference from the 1.9 percentage point reduction for non-NGSI students. 

Table 4.28. Retention Rates for Students Repeating Ninth Grade, by Economic Indicators 



NGSI Students 

Non-NGSI Students 

Number of 
Students 

Percent 

Retained 

Number of 
Students 

Percent 

Retained 

Economically 

disadvantaged 

1999-00 

827 

33.4 

827 

36.3 

2000-01 

2,090 

31.9 

2,090 

31.3 

2001-02 

2,192 

20.2 

2,192 

23.9 

2002-03 

1,534 

27.2 

1,534 

29.5 

Non-economically 

disadvantaged 

1999-00 

508 

33.7 

508 

29.5 

2000-01 

1,468 

30.4 

1,468 

30.3 

2001-02 

1,349 

18.8 

1,349 

22.8 

2002-03 

1,066 

25.6 

1,066 

27.6 

Reduction in 
retention“ 

Economically 

disadvantaged 

- 

-6.2 

- 

-6.8 

Non-economically 

disadvantaged 

- 

-8.1 

- 

-1.9 


“Difference between 1999-00 and 2002-03. 
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Retention by Ethnicity 

> Although Hispanic and African American students served in NGSI programs had the highest 
retention rates, these students also had the greatest reductions in retentions across program 
years. 

Ninth grade retention rates, disaggregated by ethnicity, are presented in Table 4.29 for NGSI and non- 
NGSI students. Hispanic and African American NGSI students had the highest retention rates, although 
rates did decline across time. Between 1999-00 and 2000-03, the retention rate decreased 7.7 percentage 
points for Hispanic students (from 33.0% to 25.3%) and 9.4 percentage points for African American 
students (33.9% to 24.5%). 

Retention rates for non-NGSI students are lower than comparable NGSI groups. Similar to NGSI 
students, the highest retention rates in 1999-00 are for Hispanic and African American students. However, 
unlike the NGSI served students, the rate of change across time is quite small for most ethnic groups, with 
the largest decline being 4.1 percentage points for Hispanic students. 


Table 4.29. Retention Rates of All Ninth Graders, by Ethnicity 



NGSI Students 

Non-NGSI Students 

Number of 
Students 

Percent 

Retained 

Number of 
Students 

Percent 

Retained 

Hispanic 

1999-00 

7,326 

33.0 

7,326 

23.0 

2000-01 

16,055 

27.3 

16,055 

19.3 

2001-02 

14,557 

26.2 

14,557 

18.0 

2002-03 

13,930 

25.3 

13,930 

18.9 

African American 

1999-00 

2,769 

33.9 

2,769 

21.3 

2000-01 

5,579 

28.8 

5,579 

20.0 

2001-02 

5,164 

25.3 

5,164 

20.2 

2002-03 

4,695 

24.5 

4,695 

20.6 

White 

1999-00 

3,891 

20.1 

3,891 

7.8 

2000-01 

9,076 

17.7 

9,076 

7.0 

2001-02 

8,112 

17.4 

8,112 

7.0 

2002-03 

8,878 

15.6 

8,878 

6.6 

Reduction in 
retention^ 

Hispanic 

— 

-7.7 

— 

-4.1 

African Am. 

- 

-9.4 

- 

-0.7 

White 

- 

-4.5 

- 

-1.2 


“Difference between 1999-00 and 2002-03. 
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Association of Student- and District-Levei Factors 
with Academic Achievement 

The relationships between NGSI student and district characteristics and reading and mathematics scores 
on state-level assessments were explored using hierarchical linear modeling (HLM). HLM was the 
preferred analytical method because, in most cases, students within school districts resemble each other 
due to selection processes and common backgrounds. Thus, measures of students (i.e., TAAS/TAKS 
scores) from the same school district are not independent, meaning that they show a greater correlation 
than do measures of students from different districts. Not only does HLM make no assumption about 
independence, it estimates the degree to which measures are dependent, and it uses this estimate in the 
calculation of the precision with which treatment effects are estimated. 

Separate HLM analyses were conducted using NGSI participants in 2000-01 (cohort 2) and 2001-02 
(cohort 3). Cohort 2 included 21,565 students from 143 school districts, and cohort 3 included 13,110 
students in 86 districts. These students participated in funded programs in their respective NGSI year, 
constituted at least 10 students in the school district, and had achievement scores from the year prior to 
NGSI participation (2000 for cohort 2 and 2001 for cohort 3) and the year after NGSI participation (2002 
for cohort 2 and 2003 for cohort 3). These students were used to investigate the effect of NGSI student 
and district characteristics on TAAS reading and mathematics TLI scores (cohort 2) and TAKS English 
language arts (ELA) and mathematics percentile scores (cohort 3). 

The specific student- and district-level variables along with their descriptive statistics are reported in 
Table 4.30. Student-level variables included gender (1 if female, 0 if male), economic status (1 if 
disadvantaged, 0 if not), prior retention (1 if retained the year prior to NGSI, 0 if not), school attendance 
(average rate for NGSI participation year), TAAS reading and mathematics TEI pretest scores,^ days 
participated in NGSI, and minority status (1 minority, 0 if white). 

District-level variables included the district NGSI mean TAAS pretest score (achievement contextual 
effect), the district proportion of disadvantaged NGSI students (economically disadvantaged contextual 
effect), the district per-pupil NGSI expenditure for that year, and the average number of days NGSI 
programs were offered. 


^ TAAS pretest scores are from the prior school year (2000 for cohort 2 or 2001 for cohort 3) except for students 
repeating ninth grade. Because TAAS was not given in the ninth grade, repeat ninth graders last took a TAAS exam 
two years earlier, in eighth grade. Therefore, TAAS pretest scores for repeat ninth graders in cohort 2 are from 1999 
and scores for cohort 3 are from 2000. 
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Table 4.30. Descriptive Statistics for TAAS/TAKS Reading and Mathematics 


Variable Name 

N 

Mean 

SD 

Minimum 

Maximum 

Cohort 2 (2000-01) Student-Level Descriptive Statistics 

Gender 

21,565 

0.49 

0.50 

0 

1 

Minority 

21,565 

0.71 

0.45 

0 

1 

Economically disadvantaged 

21,565 

0.49 

0.50 

0 

1 

Prior retention 

21,565 

0.06 

0.23 

0 

1 

School attendance 

21,565 

96.00 

4.12 

38.60 

100.00 

TAAS reading (prior) 

21,565 

87.77 

12.16 

11.00 

99.00 

TAAS reading 2002 

21,565 

85.95 

9.68 

5.00 

97.00 

TAAS mathematics (prior) 

21,565 

79.97 

9.23 

7.00 

92.00 

TAAS mathematics 2002 

21,565 

80.71 

8.43 

20.00 

92.00 

Days participated in NGSI 

21,565 

92.94 

59.48 

0 

179.00 

Cohort 2 (2000-01) District-Level Descriptive Statistics 

Economically disadvantaged 

143 

53.72 

21.40 

10.77 

98.36 

Reading and math achievement 
(prior) 

143 

77.39 

4.96 

60.92 

89.30 

NGSI expenditure per pupil 

143 

624.12 

1,136.44 

56.00 

12,702.00 

Days NGSI offered 

143 

88.30 

40.93 

4.59 

175.15 

Cohort 3 (2001-02) Student-Level Descriptive Statistics 

Gender 

13,110 

0.50 

0.50 

0 

1 

Minority 

13,110 

0.78 

0.41 

0 

1 

Economically disadvantaged 

13,110 

0.62 

0.49 

0 

1 

Prior retention 

13,110 

0.03 

0.18 

0 

1 

School attendance 

13,110 

95.51 

4.49 

21.05 

100.00 

TAAS reading (prior) 

13,110 

83.86 

11.92 

11.00 

99.00 

TAKS ELA 2003 

13,110 

35.35 

25.54 

0.00 

99.99 

TAAS mathematics (prior) 

13,110 

80.30 

8.49 

8.00 

92.00 

TAKS mathematics 2003 

13,110 

36.13 

25.22 

0.07 

99.77 

Days participated in NGSI 

13,110 

149.76 

49.71 

0 

179.0 

Cohort 3 (2001-02) District-Level Descriptive Statistics 

Economically disadvantaged 

86 

57.73 

21.20 

11.68 

100.0 

Reading and math achievement 
(prior) 

86 

80.02 

4.95 

65.36 

90.00 

NGSI expenditure per pupil 

86 

429.93 

395.55 

56.00 

3059.00 


Student-Level Variables 

> After controlling for the effect of student-level characteristics (academic and social 
background), there was no positive relationship between the number of days students spend 
in NGSI and their achievement scores. 

> A student’s school attendance rate was positively associated with both reading and 
mathematics achievement. 

Table 4.31 shows that student-level predictors associated with higher reading/ELA scores include being 
economically advantaged, female, and non-minority. Higher TAAS reading pretest scores were also 
strongly associated with higher posttest scores for TAAS reading and TAKS ELA. Eor example, for 
cohort 2 NGSI students, a unit increase in 2000 TAAS reading scores was associated with a 0.50 unit 
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increase in 2001 TAAS reading scores. School attendance was also a significant predictor of reading and 
ELA achievement. By way of example, a 1% increase in attendance rate resulted in a 0.90 unit increase in 
2001 TAAS reading posttest scores, net of other student-level predictors. 

A negative relationship existed between NGSl instructional days and TAAS reading TLl scores in 
cohort 2. This relationship might he reasonable if poorly performing students received more days of 
instruction than better performing students. For cohort 3 students, there was no significant relationship 
between the number of NGSl instructional days that students participated and TAKS scores for ELA. 

Similar student-level predictors also were associated with higher TAAS/TAKS mathematics scores, 
though female students had lower mathematics scores than male students. As with the reading analyses, a 
negative relationship existed between the number of days students participated in NGSl and TAAS 
mathematics scores for cohort 2 students. For cohort 3 students, there was no significant relationship 
between NGSl instructional days and TAKS mathematics scores. 


Table 4.31. HLM Analyses of TAAS Reading and Mathematics TLl Scores for Cohort 2 (2000- 
00) and TAKS ELA and Mathematics Percentile Scores for Cohort 3 (2001-02) NGSl Students 


Conditional Model 

Reading / ELA 

Mathematics 

Cohort 2 

Cohort 3 

Cohort 2 

Cohort 3 

Student Level Variables 

District Level Variables 

Gamma 

Coefficient/ 

if) 

Gamma 

Coefficient/ 

if) 

Gamma 

Coefficient/ 

if) 

Gamma 

Coefficient/ 

if) 

Intercept 


86.93 

(334.36**) 

35.23 

(22.22**) 

82.07 

(317.38**) 

38.03 

(29.43**) 


Economically 
disadvantaged context 

0.005 

(0.55) 

0.03 

(0.87) 

0.012 

(1.53) 

0.03 

(1.32) 


Achievement context 

0.05 

(1.10) 

0.11 

(0.81) 

0.058 

(1.33) 

0.78 

(0.63) 


NGSl per-pupil 
expenditure 

0.0001 

(1.86) 

0.001 

(1.16) 

0.0001 

(1.39) 

-0.002 

(0.89) 


NGSl program days 

0.013 

(3.49**) 

n/a 

0.020 

(5.48**) 

n/a 

Female 


-0.18 

(1.84) 

6.19 

(13.01**) 

-0.63 

(5.60**) 

-0.48 

(1.72) 

Minority status 


-1.13 

(8.01**) 

-3.31 

(2.68*) 

-1.05 

(5.73**) 

-4.82 

(4.04*) 

Economically 

disadvantaged 


-0.77 

(6.08**) 

-2.59 

(4.48**) 

-0.37 

(3.14*) 

-2.42 

(6.37**) 

Repeating current year 


0.26 

(0.67) 

-3.54 

(1.97) 

0.49 

(1.34) 

-5.66 

(2.72*) 

School attendance rate 


0.90 

(5.45**) 

0.53 

(7.47**) 

0.12 

(11.65**) 

0.75 

(10.76**) 

TAAS pretest 


0.50 

(58.11**) 

1.06 

(33.62**) 

0.62 

(44.85**) 

1.51 

(21.16**) 

Days participated in NGSl 


-0.007 

(3.67**) 

-0.01 

(1.38) 

-0.01 

(9.59**) 

0.003 

(0.47) 


Notes. In the student-level model, the intercept was specified as random and the independent variables were 
specified as fixed. The number of NGSl program days was unavailable for cohort 3 (“n/a”). 

*p< .01; **p< .001. 
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District-Level Variables 

> There was a slightly positive association between the average number of NGSI program days 
offered and academic achievement for TAAS reading and math. 

> There was no significant relationship between NGSI dollars per pupil and TAAS/TAKS scores. 

Contextual effects occur when the aggregate of a student-level characteristic is related to an outcome 
variable, even after controlling for the effect of the student-level characteristic. An example would he 
average social class of a school being related to achievement after controlling for individual students’ 
social class (Raudenbush & Bryk, 2002). HLM facilitates this type of multi-level analysis by specifying 
separate student and district equations. Following this model, Table 4.31 shows that there was a slightly 
positive association between the average number of NGSI program days offered and subsequent TAAS 
reading and mathematics scores (a variable for NGSI program days was unavailable for cohort 3). 
Conversely, the average NGSI dollars spent per student did not have a significant effect on district 
reading/ELA or mathematics scores. 

Association of Student-and District-Levei Factors 
with Retention 

Relationships between NGSI student and district characteristics and retention status (a binary outcome, 
retained or not retained) were analyzed using a hierarchical generalized linear model (HGLM) with a 
Bernoulli sampling model, a log odds or logit link function, and level- 1 (student) and level-2 (district) 
structural models identical to those in HLM. HGLM presents results for both unit-specific and 
population-average models. The unit-specific model holds constant school district attended, while the 
population- average model does not, but averages over all districts. Because the average log-odds of 
retention was found to vary significantly across districts this variation should be controlled or held 
constant. Consequently, only unit-specific results will be presented and discussed below. (Note, however, 
that results are similar for both models.). 

Included in separate HGLM analyses were students who participated in NGSI in 2000-01 (cohort 2) and 
2001-02 (cohort 3). Cohort 2 included 30,284 NGSI students from 146 school districts, whereas cohort 3 
included 18,033 students from 90 school districts. These data were used to investigate the effect of NGSI 
student and district characteristics on retention status (2001 retention status for cohort 2 and 2002 
retention status for cohort 3). See Table 4.32. 

As previously defined, student characteristics included gender, economic status, prior retention, days 
participated in NGSI, and minority status. School attendance in the year of NGSI participation and the 
average of the TAAS reading and mathematics TLI scores in the prior year of participation were also used 
as student-level predictors. 

District-level variables included the district NGSI mean of the TAAS combined reading and mathematics 
score (achievement contextual effect), the district proportion of disadvantaged NGSI students, and the 
district NGSI per pupil expenditure for the appropriate cohort year. 
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Table 4.32. Descriptive Statistics for Student Retention 


Variable Name 

N 

Mean 

SD 

Minimum 

Maximum 

Cohort 2 (2000-01) Student-Level Descriptive Statistics 

Gender 

30,284 

0.48 

0.50 

0 

1 

Minority 

30,284 

0.73 

0.44 

0 

1 

Economically disadvantaged 

30,284 

0.51 

0.50 

0 

1 

Prior retention 

30,284 

0.10 

0.30 

0 

1 

Retention 2001 

30,284 

0.20 

0.40 

0 

1 

School attendance 

30,284 

94.04 

7.43 

11.63 

100.0 

Combined prior TAAS score 

30,284 

79.76 

11.18 

9.00 

95.50 

Days participated in NGSI 

30,284 

90.77 

58.73 

0 

179.00 

Cohort 2 (2000-01) District-Level Descriptive Statistics 

Economically disadvantaged 

146 

53.99 

21.46 

10.77 

98.36 

Reading and math achievement 
(prior) 

146 

77.35 

4.92 

60.92 

86.30 

NGSI expenditure per pupil 

146 

628.14 

1,127.56 

56.00 

12,702.00 

Days NGSI offered 

146 

88.24 

40.71 

4.59 

175.15 

Cohort 3 (2001-02) Student-Level Descriptive Statistics 

Gender 

18,033 

0.48 

0.50 

0 

1 

Minority 

18,033 

0.80 

0.40 

0 

1 

Economically disadvantaged 

18,033 

0.64 

0.48 

0 

1 

Prior retention 

18,033 

0.10 

0.30 

0 

1 

Retention 2001 

18,033 

0.19 

0.39 

0 

1 

School attendance 

18,033 

93.66 

7.16 

17.02 

100.00 

Combined prior TAAS score 

18,033 

80.19 

10.43 

9.50 

95.50 

Days participated in NGSI 

18,033 

147.04 

49.51 

0 

179.00 

Cohort 3 (2001-02) District-Level Descriptive Statistics 

Economically disadvantaged 

90 

58.70 

21.38 

11.68 

100.00 

Reading and math achievement 
(prior) 

90 

78.70 

5.84 

44.00 

90.06 

NGSI expenditure per pupil 

90 

456.39 

435.14 

56.00 

3059.00 


Student-Level Variables 

> After controlling for the effect of student-level variables (academic and social background), 
more instructional days in NGSI were associated with a decreased chance of retention for 
both student cohorts. 

> For otherwise similar students, an increase in a student’s school attendance rate decreased 
the chance of retention. 

For both student cohorts, almost all student-level variables exerted a statistically significant effect on the 
probability of a student being retained (see Table 4.33). Being female, failing to be promoted the previous 
school year, and high TAAS pretest scores were all associated with a lower probability of being retained. 
More instructional days in NGSI also decreased the chances of retention. An increase in a student’s 
school attendance rate was also associated with a reduced chance of being retained. As one might expect, 
being non-White or economically disadvantaged was associated with a higher probability of retention. 
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Based on the HGLM analysis to predict retention, the prohahility that the average student in cohort 2 was 
retained at the end of ninth grade is 12.4% and for cohort 3 students, 9.6%.^ The addition of 20 NGSI 
service days decreases the predicted prohahility of cohort 2 student retention to 12.0% and the prohahility 
for cohort 3 to 8.8%. In comparison, a 1% change in a student’s school attendance rate would decrease 
the predicted prohahility of retention for cohort 2 students to 10.9% and cohort 3 to 8.4%. 


Table 4.33. HGLM Analyses of Retention Status for Cohort 2 (2000-01) 
and Cohort 3 (2001-02) NGSI Students 


Conditional Model 

Cohort 2 

Cohort 3 

Student Level Variables 

District Level Variables 

Gamma 

Coefficient/ 

(t) 

Gamma 

Coefficient/ 

(t) 

Intercept 


-1.64 

(16.33**) 

-1.18 

(6.49**) 


Economically 
disadvantaged context 

-0.003 

(0.69) 

0.002 

(0.28) 


Achievement context 

-0.05 

(2.60*) 

-0.04 

(1.45) 


NGSI per pupil 
expenditure 

-0.0004 

(2.41*) 

-0.0008 

(2.23*) 


NGSI program days 

0.002 

(0.82) 


Female 


-0.44 

(12.67**) 

-0.56 

(12.41**) 

Minority status 


0.19 

(3.85**) 

0.14 

(1.87**) 

Economically 

disadvantaged 


0.12 

(2.88*) 

0.20 

(3.67*) 

Repeating current year 


-1.19 

(18.84**) 

-1.64 

(19.10**) 

School attendance rate 


-0.15 

(50.44**) 

-0.15 

(38.33**) 

TAAS pretest 


-0.05 

(33.07**) 

-0.06 

(26.28**) 

Days participated in NGSI 


-0.002 

(3.40**) 

-0.005 

(4.78**) 


Note. Student-level predictors were specified as fixed with error terms constrained to 
be 0. Intercepts varied randomly. 

*p< .01; **p< .001. 


^ Probabilities were calculated using the function; p = e**^’‘* / (1 + e**^’'*) where is first calculated by using all 
coefficients in Table 4.32 and coding all variables at the mean. All continuous variables without a true zero were 
grand mean centered and thus their mean is zero. 
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District-Level Variables 

> After controlling for the effect of student characteristics (academic and social background) 
and district social and academic contexts, higher NGSI per-pupil expenditures was associated 
with reduced student retention. 

> For cohort 2 students, having higher achieving classmates was associated with a reduced 
chance of retention. 

District-level contextual variables were less likely to exert a significant effect. For both student cohorts, 
being in a school that expended relatively greater amounts of money per student was associated with a 
lower probability of retention. For cohort 2 students, being in a school with higher achieving students was 
associated with a lower probability of retention. Although variables were found to be statistically 
significant, they still accounted for only a moderate change in the probability of someone being retained. 
Using cohort 2 as an example, adding $100 to district NGSI per-pupil expenditure would result in a 0.4 
percentage point decrease in the predicted probability of being retained. The larger impact is seen for 
students from high achieving schools. Students attending a school with achievement scores one standard 
deviation above the norm have a 2.8 percentage point decrease in the predicted probability of being 
retained. 


Ill 



CONCLUSIONS AND IMPLICATIONS 


From 1999 to 2002, the state of Texas appropriated a total of $170 million for the Basic Skills Program 
for High School Students — also known as the Ninth Grade Success Initiative (NGSl) — to support school 
districts’ efforts to help ninth graders stay in school and succeed academically. The goal of NGSl was to 
increase graduation rates in Texas public schools hy reducing the number of students who were retained 
in or dropped out of the ninth grade. Funds went toward expanding or enhancing existing programs, or 
creating new programs to increase academic performance and attendance rates and reduce dropout rates 
for ninth graders who had not earned — or were unlikely to earn — sufficient credit to advance to tenth 
grade or eighth graders who were promoted but considered academically at risk. The purpose of this study 
was to assess the effective use of grant funds by 226 school districts receiving both original and 
continuation NGSl funding during four school years (1999-00 through 2002-03). Conclusions and 
implications relative to the study’s research questions are summarized below. 


What are the characteristics of districts and students receiving NGSl grants? 


> Characteristics of districts. NGSl district size distributions differ from the state overall, in which 
the majority of districts have very small enrollments (fewer than 1,000 students). More than half 
(58%) of NGSl grant recipients were midsize to very-large districts (3,001 to more than 25,000 
students). Compared to the state, high schools in NGSl districts also had a slightly greater proportion 
of minority, limited English proficient, and economically disadvantaged students. 

^ Characteristics of students. During four program years (1999-00 to 2002-03), 389,834 unique 
students participated in the NGSl program (based on an estimated count that adjusts for missing data). 
The number of NGSl students participating in program activities varied by year, increasing from 
32,535 (spring 2000) to 106,325 (2002-03). Substantially fewer students participated during summer 
terms, but enrollments climbed steadily during the grant period (from 19,508 to 31,607). 

The majority of NGSl students (80% or more during regular terms) were ninth-grade students at-risk 
of not earning sufficient credits to advance to tenth grade. About three-fourths of NGSl students were 
Hispanic (about 56%) and African American (about 17% during regular terms and 20% during 
summer). The majority of students served by NGSl programs were in the ninth grade for the first time 
(80% or more each term). The percentages of newly promoted ninth graders served in NGSl declined 
across summer terms (from about 33% to 9%). 


How did grant resources supplement existing educational programs? 


> Program characteristics. On average, NGSl programs had larger staff, more volunteers, and 
higher student-to-teacher ratios for fall and spring semesters compared to summer terms. However, 
since NGSl programs served fewer students in summer terms, mean student-to-teacher ratios in the 
summer (10.2 to 13.0) were substantially lower than regular terms (23.2 to 27.4). 

> Activities supporting credit recovery and basic skills. NGSl programs typically used several 
activities to serve at-risk ninth graders. Tutoring, instructional technology, individual instruction, 
group instruction, and counseling were reported most often. The diversity of approaches reported for 
programs makes it difficult to assess effective practice. 

> Course credit accrual and instructional methods. During the regular school year, students 
accrued course credit primarily through regular classroom instruction, but repeat ninth graders were 
more likely to accrue credit through computer-aided instruction (e.g., PLATO or NovaNET self- 
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paced learning systems). NGSI instructional methods changed during the course of the grant, with the 
use of tutoring decreasing and the use of counseling and mentoring increasing. 

> Successful program features. Technology-hased instruction and learning emerged as the most 
consistently reported successful program feature. According to many, self-paced programs such as 
PLATO and NovaNET benefited students hy individualizing instruction, meeting various learning 
styles, allowing a flexible pace, supporting the acquisition of credits to stay on grade level, and 
providing an alternative to traditional educational settings. Although tutoring was cited as the most 
successful feature for regular terms, it was viewed less consistently as a successful feature of summer 
programs. This is probably because lower student-to-teacher ratios in the summer reduced the need 
for tutorials. 


What is the effect of grant resources on targeted students? 


> Core-content courses. Passing rates for core subject-area courses remained relatively stable across 
NGSI grant terms, with about 70% of NGSI students passing Algebra I during regular terms and 
about three-fourths or more of students passing Biology, Integrated Physics and Chemistry (IPG), 
World Geography, and English I. Course passing rates increased for summer terms, with the majority 
of NGSI students who participated in summer terms passing core courses (about 80% to 95%). 
However, the number of students enrolled in summer terms decreased substantially. 

As might be expected, students in ninth grade for the first time had higher passing rates for core- 
subject courses than students who did not earn sufficient credits for promotion. Newly promoted ninth 
grade students who participated in summer coursework also had high passing rates (89% to 98%); 
however, only a few students participated. 

> Attendance. In general, NGSI students’ attendance rates did not improve across grant terms. Eor 
both NGSI and a comparison group of non-NGSI students, first-time ninth graders had substantially 
higher attendance rates (about 92% to 96%) than repeat ninth graders (about 83% to 93%). 

NGSI first-time ninth graders had slightly lower attendance rates than their non-NGSI peers (about 
0.5 to 2.0 percentage points). Attendance rates for repeat NGSI ninth graders, however, were typically 
near or surpassed non-NGSI comparison groups. Although attendance rates for both first-time and 
repeat NGSI students declined across time, this change was reduced slightly during the NGSI 
program implementation year for students repeating ninth grade. 

> State-level assessments. NGSI students had lower TAAS passing rates for both reading and math 
compared to non-NGSI students, but the achievement gap between groups narrowed (to 3.7 points in 
reading and 6.6 points in math). Despite encouraging results for TAAS, the achievement gap widened 
substantially for students in cohorts 3 and 4 who completed the TAKS (to about 18 percentage points 
for math). NGSI repeat ninth graders had similar TAAS and TAKS passing rates compared to non- 
NGSI students for both reading and mathematics. However, for both student groups, passing rates 
declined substantially for TAKS reading (70% and 72%) and math (35% and 39%). 

> Retention rates. Although NGSI student retention rates remain high (21.8% in 2002-03), evidence 
for four program years reveals that NGSI retention rates have decreased more than rates for non- 
NGSI students (-7.7 points compared to -3.3). Eirst-time ninth graders had greater declines in 
retention rates than non-NGSI students (-7.6 points compared to -3.1). Retention rate declines were 
similar for economically disadvantaged and advantaged students. 

Although Hispanic and African American students served in NGSI programs had the highest retention 
rates (25% in 2002-03), these students also had the greatest reductions in retentions across program 
years (-7.7 and -9.4 points, respectively for Hispanic and African American students). 
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What program elements are associated with student outcomes? 


To further explore the association between NGSI student and district characteristics and TAAS reading 
and mathematics TLI scores and TAKS English language arts and mathematics percentile rank scores, 
researchers used hierarchical linear modeling (HEM). Separate HEM analyses were conducted using 
NGSI participants in 2000-01 (cohort 2) and 2001-02 (cohort 3). Cohort 2 included 21,565 students from 
143 districts, while cohort 3 included 13,110 students from 86 districts. These students participated in 
NGSI programs in their respective districts. Separate HEM analyses were also conducted for retention. 
The numbers of students and districts available for analyses included 30,284 students in 146 districts for 
cohort 2 and 18,033 students in 90 districts for cohort 3. 

Student-Level Variables and Outcomes 

> After controlling for the effect of student-level characteristics (academic and social background), 
there was no positive relationship between the number of days students participated in NGSI and 
achievement scores on TAAS/TAKS reading/EEA or mathematics. In fact, a negative relationship 
existed between days and TAAS achievement. This relationship might be reasonable if poorly 
performing students received more days of instruction. In contrast to outcomes for student 
achievement, more instructional days in NGSI were associated with a slightly decreased probability 
of retention for student in both cohorts (2000-01, 2001-02). 

> Ninth graders’ school attendance was an important predictor of academic performance. Eor otherwise 
similar students, a student’s school attendance rate was positively associated with both TAAS/TAKS 
reading/EEA and mathematics achievement. Moreover, for both student cohorts an increase in a 
student’s school attendance rate was associated with a decreased chance of retention. 

> After controlling for student-level characteristics, NGSI students’ academic achievement varied 
significantly by district. This suggests that some districts and schools were more successful in 
meeting the needs of ninth graders. 

District-Level Variables and Outcomes 

> Based on NGSI program information available for cohort 2 students, there was a slightly positive 
association between the number of NGSI program days offered and TAAS reading and math scores. 
Thus, districts that made a larger number of days available for the program had greater success in 
improving student achievement. 

> Similar to findings for the Optional Extended Year program and Texas After School Initiative, there 
was no significant relationship between NGSI dollars per pupil and student achievement. However, 
for NGSI, higher per-pupil expenditures were associated with reduced student. 

> Eor students in cohort 2, having higher achieving classmates was associated with a slightly reduced 
chance of retention. 


What are the implications for addressing the needs of students at risk? 


> Few districts designed programs for newiy promoted ninth graders who iacked minimum 
skiiis for successfui course compietion. The majority of students served by NGSI programs were 
in the ninth grade for the first time (80% or more each term). The percentages of newly promoted 
ninth graders served in NGSI declined across summer terms (from about 33% to 9%). Even though 
NGSI funds could be used to proactively address the needs of eighth graders before high school 
enrollment, this approach was seldom used. 
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> African American students were more iikeiy to be enroiied in summer schooi programs, which 
typicaiiy heiped students to recover credits for faiied courses, rather than program 
interventions during reguiar schooi terms. Student enrollment trends for NGSI terms revealed 
differing enrollment patterns by ethnic groups. In particular, trends for African American students 
showed the percentage of students in NGSI programs increased during summer terms. This suggests 
that interventions for African American students focused more often on remediation of academic 
failure rather than proactive efforts to improve success. 

> Evidence from district NGSi reports shed iittie iight on the identification of effective programs 
for students at risk. NGSI programs essentially remain a “black box.” The TEA attempted to collect 
a great deal of information on the nature of NGSI instructional programs through program and 
activity reports. However, the instructional and learning focus of programs remains unclear because 
the majority of districts reported using multiple approaches. For example, during regular terms, high 
percentages of districts reported that their program featured activities such as tutoring (about 85%), 
instructional technology (about 75%), individual instruction (about 70%), and instruction in the 
regular classroom (about 65%). It is impossible to determine the effectiveness of a program when 
students are apparently receiving multiple interventions. 

> Students repeating ninth-grade coursework were more iikeiy to accrue credit through seif- 
paced computer-aided instruction. However, there is iittie evidence to support program 
effectiveness. Districts invested a substantial proportion of NGSI grant resources in instructional 
technology. In particular, many districts reported the implementation of PLATO or NovaNET self- 
paced instructional systems in labs for tutorials, credit recovery, or credit accrual. Findings show that 
ninth graders who did not earn sufficient credits for promotion were more likely than first-time ninth 
graders to utilize such programs for coursework. According to district reports, self-paced programs 
were highly successful in individualizing instruction, supporting the acquisition of credits, providing 
an alternative means to deliver instruction, and so forth. Despite positive perceptions, little is known 
about the effectiveness of self-paced programs that are being widely used to address the needs of at- 
risk students. Available evidence suggests that such programs may reduce student retention but the 
impact on student achievement is suspect. 

> Improving performance in core-subject area coursework is critical to the success of at-risk 
students. One important indicator of ninth-grade success would be improved passing rates in core- 
subject areas. However, findings for the three full grant implementation years show that NGSI had no 
positive effect on ninth graders’ course passing rates for Algebra I, Biology, IPG, World Geography, 
or English I. About 70% or less of students passed Algebra I each year and about three-fourths of 
ninth graders passed other courses. Such low passing rates mean that about 25% to 30% of NGSI 
ninth graders remain at risk of not being promoted to tenth grade. Algebra I is a major obstacle for 
many students, with nearly 30% of first-time and 40% of repeat ninth graders failing algebra. Clearly, 
meeting the needs of at risk students will require the improvement of student performance in core 
courses. 

> Poor school attendance continues to jeopardize the academic success of students at risk. A 

key goal of the NGSI program was the improvement of student attendance. Results show, however, 
that NGSI had no positive, sustained effect on ninth graders’ school attendance. More importantly, 
attendance rates for both first-time and repeat ninth graders decline as they progress to higher grade 
levels. Although there was a slight improvement in the attendance rate for repeat ninth graders during 
the NGSI year, gains were not sustained into the next school year. Findings regarding attendance are 
particularly important because school attendance emerges as a significant predictor of student 
academic achievement and reduced chances of retention. Future initiatives supporting students at risk 
should directly address strategies to increase high school attendance. 
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> Modest accomplishments for the NGSI suggest that the $170 million invested in the initiative 
did not achieve program goals for students at risk. Overall findings for the NGSI suggest that, as 
a whole, the program was somewhat effective in reducing ninth-grade retention rates hut fell short of 
accomplishing other important goals such as improved attendance and increased academic 
achievement on state-level assessments. Tracking reductions in student dropout was heyond the scope 
of this study. Although we did not conduct a thorough study of cost effectiveness, analyses revealed 
no significant associations between NGSI per-pupil expenditures and the academic achievement of 
students at risk. Per-pupil expenditures were associated with a slightly reduced prohahility of student 
retention, hut the effect may not justify the cost. 
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Chapter 5 

CASE STUDIES OF INITIATIVES SUPPORTING NINTH GRADERS’ SUCCESS 


Researchers conducted case studies of Ninth Grade Success Initiative (NGSI) grants to gain a greater un- 
derstanding of issues facing large numbers of at-risk students, many of whom, despite potentially receiv- 
ing services as early as kindergarten, still reach ninth grade unprepared to succeed academically in high 
school. Many of these students end up repeating ninth-grade coursework or dropping out of school. Case 
studies focused on NGSI projects and the broader high school contexts in which they operated. Research- 
ers conducted intensive studies in 11 of 226 districts that received NGSI funding between 1999-2000 and 
2002-03. In addition to NGSI funds, districts also benefited from Optional Extended Year program 
(OEYP) formula-based allocations, Texas After School Initiative (TASI) grants, or both. 

Case Studies of NGSI Grantees 

Case studies give an in-depth look at district- and campus-level activities supporting students in at-risk 
situations, grant- funded activities sustained over time, and best practices in projects. Researchers were 
guided by four overarching research questions: 

• How was the NGSI program implemented and what was the effect of grant resources on targeted stu- 
dents, 

• How did grant initiatives intersect with the broader ninth-grade context, 

• How did the educational environment in high schools support grant goals for students at risk, and 

• What are the implications for addressing the needs of students at risk? 

Site Selection 

Case studies include 10 single district grantees and one consortium (representing two districts). Site selec- 
tion for case studies was a multi-stage process. Researchers selected districts that implemented programs 
of sufficient scope to have a potentially measurable impact on a significant number of students. We re- 
viewed activity/progress reports submitted by 226 NGSI grantees receiving both original and continuation 
funds to create a database with key indicators (e.g., budget allocations, targeted populations, grant focus, 
etc.). This list was narrowed by including only districts that had (a) more than 50 percent economically 
disadvantaged students, (b) implemented a program targeting more than 25 students and at least 20 per- 
cent of the ninth-grade population, (c) a grant allocation in excess of $50,000 per year, and (d) a begin- 
ning ninth-grade retention rate above 10 percent. Districts with missing data on relevant variables were 
eliminated. Erom the resulting list of 57 districts, researchers in consultation with TEA staff members 
chose 11 districts, with careful consideration given to diversity. Thus, selected districts represent diverse 
regions of the state, varied demographic and grant characteristics, and distinctive program aspects consid- 
ered worthy of investigation (See details in Appendix D.) 

Data Collection Methods 

Teams of two to three researchers conducted site visits to each of the 11 case-study sites. Site visits in- 
cluded structured interviews, focus groups, surveys, and classroom observations designed to collect in- 
formation about the primary research questions. During visits, researchers also observed NGSI-supported 
activities and collected relevant materials and documents. 

Interviews. A total of 47 interviews involved targeted district and campus staff, including the project 
director, principal, onsite project coordinator, a lab facilitator, and other staff depending on the character- 
istics of the program implemented. 
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Focus groups. Researchers conducted 26 teacher focus groups involving 124 teachers at 16 high 
schools. Focus groups consisted of teachers involved with the NGSI program and other randomly selected 
ninth-grade teachers. We also conducted 36 student focus groups with 202 ninth- and tenth-grade stu- 
dents. At each school, at least one focus group consisted of students who had participated in NGSI activi- 
ties in either the current or previous year, and one focus group included ninth graders in at-risk situations. 

Surveys. Teachers providing instruction to ninth graders were asked to complete a questionnaire solicit- 
ing their opinions on the high school environment. Out of 563 surveys distributed, 283 were returned (50 
percent response rate). Of these, 124 teachers completed questionnaires during focus groups, and 159 re- 
turned them by mail. The 202 students participating in focus groups also completed a brief questionnaire 
assessing their views on the school environment and plans for the future. 

Observations. Across all campuses, researchers observed in 92 classrooms, including 81 regular class- 
rooms and 1 1 computer laboratories. This sample of core-subject area classrooms was selected through a 
review of at-risk students’ course schedules in each school and included 21 observations in Eng- 
lish/language arts classes, 21 in Algebra I, 16 in social studies, and 23 in science. 

A conceptual framework, formulated through a review of program objectives and recent research litera- 
ture on recommended improvements in the nation’s high schools (e.g., American Youth Policy Forum, 
2000; High Schools that Work — Frome, 2001; NASSP, 1996/2003) provided the framework for the 
study. The complete cross-site report -The Texas Study of Students at Risk: Case Studies of Initiatives 
Supporting Ninth Graders Success — is available at www.tcer.org. See the full report for a complete meth- 
odological description. 

Organization of the Chapter 

Conclusions and implications drawn from the complete report are presented below. Findings are organ- 
ized around four areas guiding the evaluation: 

• The NGSI program and existing best practices, 

• Evidence on the effect of grant resources on students, 

• Support for at-risk students within the school context and educational environment, and 

• Recommendations for grant awards and management. 


How was the NGSI program implemented and what best practices exist? 


Programs for Newiy Promoted Ninth Graders 

> Few districts offered programs for newiy promoted ninth graders who iacked minimum skiiis 
for successfui course compietion. 

> Educators beiieved newiy promoted ninth graders who participated in summer programs 
benefited from reduced ciass size, active iearning, bonding with teachers, and high schooi 
orientation. 

> Even though educators viewed summer aigebra camps and programs as worthwhiie and ef- 
fective, few students participated and most programs were discontinued. 

Research shows that students’ motivation to learn is at the heart of successful learning (American Psycho- 
logical Association, 1993). Although most students begin with an excitement for learning, enthusiasm 
declines as they progress from elementary to high school for various reasons (e.g., learning opportunities, 
interactions with teachers and peers, expectations about ability) (Weinstein, 2000). Proactive efforts to 
ensure student success (and enhance beliefs about competence) can help foster student engagement in 
learning, and therefore achievement (National Research Council & Institute of Medicine, 2004). Although 
NGSI grant recipients could design programs to meet the needs of recently promoted eighth graders, such 
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efforts generally were limited in scope and often discontinued, apparently due to a lack of student interest 
and participation. 

Statewide data confirmed this trend at case study sites (see Table 4.8). The percentage of newly promoted 
ninth graders participating in summer programs decreased across grant terms (from 33 percent in 2000 to 
9 percent in 2003). Declining emphasis on early intervention is troubling because nearly all educators be- 
lieved programs such as algebra camps benefited students. To better understand the potential of programs 
for newly promoted ninth graders, further research is needed to identify effective programs, determine 
why many students in at-risk situations fail to participate, and understand why districts and high schools 
seldom direct grant funds toward preventive programs. 

Programs for First-Time and Repeat Ninth Graders 

In contrast to the dearth of programs for newly promoted ninth graders, districts invested the bulk of 
NGSI resources in services for ninth graders who were at-risk of not earning sufficient credit or had not 
earned sufficient credit to advance to grade 10. Grant initiatives discussed below center on computer- 
assisted instruction, extended-day and extended-year programs, and whole-school improvement (restruc- 
turing, core-subject enhancement, and professional development). 

Computer-Assisted Instruction 

Most districts invested a substantial proportion of NGSI funds in technology for computer-assisted in- 
struction. Instructional technology for students considered at risk most frequently included comprehensive 
programs supporting self-paced credit recovery or skill remediation. A few districts purchased programs 
that provided comprehensive algebra coursework programs or supplemental instruction in core-subject 
areas. 

Self-Paced Credit Recovery Labs 

> Staffing of self-paced credit recovery labs for at-risk students most often involved one certi- 
fied teacher who managed student coursework in several core-subject areas. 

> One very large district took a more comprehensive approach to student credit recovery by es- 
tablishing Learning Labs with computer- and text-based assignments, instructional support, 
and social services. 

> Almost all educators and students believed self-paced courseware benefited students by of- 
fering alternative means for credit recovery, but student learning outcomes for comprehensive 
services were most promising. 

> Concerns with self-paced learning programs include software quality, TEKS and TAKS align- 
ment, student attendance, recruitment of effective teachers, and whether earned credits reflect 
content mastery. 

Districts, especially those with large-to-very large enrollments, most often established computer labs for 
credit recovery using self-paced computer-assisted instruction (PLATO or NovaNET). Self-paced credit 
recovery labs typically involved one certified teacher who managed student coursework. In contrast, one 
very large district established a Learning Lab in each high school, each staffed with four content-area 
teachers, a counselor, and a student liaison (paraprofessional). Students completed a combination of com- 
puter-assisted and other assignments (e.g., writing, problem solving). As a whole, this credit recovery 
model seemed to enhance the prospects of at-risk students for successful learning. Although nearly all 
educators believed self-paced courseware promoted credit recovery, educators and students more often 
credited the learning lab model with outcomes such as improved student self-image and confidence, read- 
ing and writing skills, and self-control and personal responsibility acquired through self-directed work. 
District outcomes verify the model’s effectiveness through improved attendance, reduced retention, and 
the continuation of labs with local funds. 
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Computer-Assisted Algebra Coursework 

> Most educators viewed the / CAN Learn and Cognitive Tutor programs positively, believing 
they helped ensure curricular consistency and improved student algebra performance. 

Two districts implemented comprehensive algebra coursework. One district invested in I CAN Learn, a 
lab-based computerized algebra curriculum, while another district purchased a program that combined 
computer- and text-based assignments {Cognitive Tutor). Most educators viewed both programs posi- 
tively, believing they helped to ensure curricular consistency and improve student performance. End-of- 
course examination results for algebra confirm educators’ opinions. Students in all participating high 
schools show strong gains, but those completing both computer- and text-based algebra assignments 
{Cognitive Tutor) had higher end-of-course passing rates. Some at-risk students voiced discontent with 
strictly computer-based algebra, preferring written work and teacher explanations instead. Overall, a 
combination of computer- and text-based learning appeared most effective in supporting students’ under- 
standing of algebra. Based on findings for computer-assisted instruction (both self-paced credit recovery 
and comprehensive coursework). Box 6.1 offers ideas for practices that appear to support effective com- 
puter-assisted coursework. 

Box 5.1. Best Practice: Comprehensive Computer-Assisted Coursework 

• Provide adequate teacher support for each core-subject area 

• Provide professional development and ongoing teacher support 

• Ensure that courseware aligns with TEKS and TAKS objectives 

• Provide a combination of computer-assisted and other assignments 

• Use performance-based assessments in addition to computer-generated tests to determine 
content mastery 

• Provide counseling and support services for at-risk students along with self-paced credit recov- 
ery coursework 

• Keep regular classroom teachers well informed about the program 

• Ensure continuity between regular course expectations and computer-assisted coursework 


Supplemental Computer-Assisted Instruction 

> Some students believe computer-assisted instruction improved learning through clear direc- 
tions, examples, and help with understanding the basics. 

> Limited access to supplemental instruction in computer labs and uneven program implemen- 
tation diminishes the potential impact on student achievement. 

One district invested in two CompassLeaming labs (English and algebra) to provide supplemental indi- 
vidualized instruction for at-risk students. Although many students spent up to 45 minutes per week 
working on computer-assisted lessons in the labs, the impact on student achievement was uncertain. 
Teachers were typically positive about the software, and some students noted learning advantages. Still, 
uneven teacher commitment to program implementation and students’ limited amount of time in labs to 
complete programs with extensive objectives diminished the prospects for a significant impact on 
achievement. 

In lieu of supplemental instruction in computer labs, high schools should consider distributing computers 
and software into classrooms to promote stronger connections between class and computer-based instruc- 
tion. This would support individualized assistance through a combination of computer-based and small- 
group instruction, as well as diagnostic and prescriptive instruction. Eor example, when one district used 
NGSI funds for computers and courseware in science classrooms, students reportedly benefited from 
online tutorials, learning from virtual experiments, remediating failed benchmark objectives, and prevent- 
ing course failure. 


120 




Extended-Day Programs 

> A few districts funded extended-day programs with tutoriais or credit recovery opportunities 
for ninth graders. 

> Students who took advantage of extended-day tutoriais apparentiy benefited, but student par- 
ticipation was a major obstacie. 

> Most students at risk are uniikeiy to attend extended-day tutoriais voiuntariiy. 

Virtually all high schools provide extended-day programs of some kind, hut five districts used NGSI 
funds for programs primarily focused on after-school tutorials. Districts configured their extended-day 
programs in varied ways: programs prepared students to recover failed coursework through credit hy ex- 
amination, teachers provided voluntary tutorials after school, or tutorials were available in lahs. As a 
whole, both educators and students generally agreed that students who took advantage of extended-day 
tutorials benefited through recovered credits, promotion, and staying with their peers. Students typically 
appreciated the one-on-one attention from teachers. 

Two districts had greater success in attracting students. In one instance, by offering extended learning 
time to complete computer-assisted work in algebra labs, and in another case, enlisting parental support 
for mandatory attendance. Overall, when student attendance was voluntary in extended-day programs, 
poor attendance was the norm. Barriers to participation included sparse access to transportation, poor pro- 
gram organization, and students who failed to see benefits. Ongoing problems with after-school programs 
led in some cases to discontinuation after funding ended. 

Educators noted similar problems with non-NGSI funded after-school programs. Although regarded as 
helpful, most at-risk students did not attend tutorials unless required to do so. Examples of successful ex- 
tended-day programs (either NGSI-funded or non-funded) were rare. Better participation, however, was 
associated with programs that were well organized and scheduled, obtained parent consent and support, 
used alternative instructional approaches (e.g., computer-assisted learning), and provided transportation. 

Extended-Year Programs (Summer School) 

> Nearly all districts used NGSI funds to provide credit recovery opportunities for ninth graders 
through summer programs. 

> Summer programs varied by duration, dally schedule, earnable credits, course delivery 
method, and core-subject availability. 

> Summer programs reportedly allowed some students to recover credits, avoid retention, and 
remain with their peers In tenth grade. 

> Districts face challenges getting ninth graders to attend summer school, ensuring regular at- 
tendance, setting high expectations for student work and behavior, and helping students pre- 
pare for subsequent coursework. 

Almost all grants studied used NGSI funds to implement summer schools or extended-year credit recov- 
ery programs for ninth graders (first-time, repeat, or both). Many districts combined local and grant re- 
sources to support programs. Districts often enhanced their summer programs by adding NGSI-funded 
instructional resources, such as self-paced courseware. Nearly all educators cited student credit recovery 
and reduced retention as summer school advantages. They also thought that keeping at-risk students on- 
grade level with their peers helped them stay in school. Smaller classes, individualized attention in sum- 
mer school, and interactive, interesting, and engaging lessons aided in student success. 

The voluntary nature of summer programs, however, narrows the population of students who attend and 
benefit. Educators point to attendance and discipline policies that eliminate disruptive or unmotivated stu- 
dents, but efforts to create a more positive learning environment also mean that many at-risk students who 
are unmotivated or have behavioral problems fail to receive much-needed academic support. Educators 
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also are challenged to ensure that students who accrue credits in summer school actually acquire the 
knowledge and skills necessary to succeed in later coursework. 

Similar to summer programs for newly promoted ninth graders, evidence from this study is insufficient to 
show how well summer schools work. However, a Southern Regional Education Board study challenges 
states that are serious about reducing student retention through summer school to establish clear standards 
for quality, program length, and scheduling of classes, and to evaluate rigorously both teaching strategies 
and student achievement (Denton, 2002). 

Whole-School Improvement 

Districts seldom used NGSI grants as an opportunity to overhaul their high schools’ approach to serving 
students at risk. However, in light of growing consensus on the need to help students cope in large, imper- 
sonal high schools, a few undertook organizational restructuring to modify instruction and services for 
ninth graders (by creating a school-within-a-school). Similarly, only a limited number of districts invested 
in core-subject course improvement or used teacher professional development to enhance classroom prac- 
tice. 

School- Within-a-School 

> In two districts, schools-within-a school provided a means to create smaller and more suppor- 
tive environments in high schools. 

> Ninth-grade teams reportedly strengthened student and teacher support, improved parent 
communication, and increased focus on student progress. 

> Some educators believe ninth graders are carrying forward organizational habits and respon- 
sible behaviors developed in the school-within-a-school. 

Some districts established schools-within-a-school (ninth-grade teams within large high schools) to im- 
prove academic achievement among students in at-risk situations. In one very large district, eight high 
schools created horizontal ninth-grade teams, but a new superintendent replaced them in the second grant 
year with vertical teams connecting groups of teachers and students in grades 9-12. Thus, this discussion 
centers on another high school that redesigned its ninth-grade program and continues to implement the 
model today with Title I funds. 

To ease students’ transition to high school, first-time ninth graders occupied one area of the school for 
most core-subject classes. Teaching teams, including an English, math, science, and social studies 
teacher, used a shared conference period to discuss student needs and parent communication. Team mem- 
bers, including an assistant principal and counselor, contacted parents of failing students to get support for 
academic improvement. Ongoing professional development also helped teachers implement teaming, un- 
derstand the unique need of students in at-risk situations, and acquire content- specific instructional strate- 
gies. 

Educators said teaming kept the focus on student success and accountability, and housing students in one 
area increased student visibility. Educators reported a dramatic change in students attending class, coming 
to class prepared, and attending tutorials. The high school also made strides in reducing the retention rate 
(from 17 to 10.5 percent) and improving academics (e.g.. Algebra I End-of-Course passing rates im- 
proved from 9 to 33 percent). Information in Box 5.2 summarizes important practices when creating a 
school-within-a-school. 
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Box 5.2. Best Practice: School-Within-a-School 

• Relocate ninth-grade classes to one area 

• Provide professional development and ongoing support for teachers 

• Focus professional development on content-specific instructional strategies 

• Provide two conference periods: one for personal planning and one for teaming 

• Include an assistant principal and counselor as team members 

• Use planning meetings to discuss student progress and needs 

• Communicate with parents regarding student progress and gain support 

• Recognize student accomplishments 


Core-Subject Course Enhancement 

> Although core-subject course enhancement occurred infrequently through NGSI grants, edu- 
cators believe initiatives improved instruction and learning. 

Grants focused on enhancing core-subject area instruction in regular classes were rare. Two districts used 
computer-assisted instruction to enhance Algebra I coursework for ninth graders. Ninth graders received 
algebra instruction via self-paced, computer-assisted instruction in I CAN Learn labs in one district. One 
high school in another district implemented Cognitive Tutor, with ninth graders working on cooperative 
problem-solving activities in classrooms and completing other lessons in a lab setting. Although research- 
ers do not endorse any particular computer-assisted program, the steps taken to improve Algebra I instruc- 
tion in one high school (as detailed in Box 5.3) are worth mentioning. 

Box 5.3. Best Practice: Enhancing Algebra Coursework 

• Adopt a program for all algebra classes (e.g., Cognitive Tutorwith text-based coop- 
erative problem solving activities and computer-assisted instruction) 

• Assign a master teacher with release time to provide oversight and mentoring 

• Provide professional development for teachers on the program 

• Align the program with district curriculum and ensure alignment across all classes 

• Assign all math teachers to at least one section of Algebra I 

• Increase class time for algebra 

• Hold weekly teacher sessions focused on instructional practices 

• Have a master teacher model instructional strategies 

• Conduct classroom observations to monitor instructional practices 

• Remove ineffective teachers from algebra classes 


Professional Development 

> Professional development was used in only a few districts as a means to improve teaching 
and learning in core-subject area classrooms. 

Many districts used NGSI funds to provide brief training sessions or workshops for teachers, especially 
on the uses of particular software programs, but few districts made intensive or sustained professional 
development for teachers a priority. Districts that attempted to improve learning in core-subject courses 
usually invested in classroom resources and training for teachers. In particular, teacher development was a 
critical component supporting the successful implementation of integrated curriculum classes, the school- 
within-a school concept, and computer-assisted algebra coursework. Findings reported in a subsequent 
section on the educational environment suggest a need for greater grant investments in teacher profes- 
sional development. 
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What was the effect of grant resources on targeted students? 


> Although research design and confounding factors made causal inferences about NGSI ef- 
fects impossible, data trends across the grant period reveal some increases in student atten- 
dance, decreases in retention rates, and improved algebra performance. 

> Despite improvements, student attendance rates are generally less than 95% (NCLB test- 
participation standard), nearly one-fifth of ninth graders are not promoted, and fewer than half 
of ninth graders typically passed end-of-course algebra exams. 

Table 5.1 reports attendance, retention, and Algebra I End-of-Course examination data for NGSI ninth 
graders by their school district and visited high school. In addition to data for 2001-02, two-year gains are 
reported (1999-00 to 2001-02). To better understand data, district NGSI gain scores are compared to state 
averages. Specifically, district retention decreases that exceeded the state two-year decrease are noted in 
bold, and district Algebra I End-of-Course examination gains that exceeded the state gain are also noted 
in bold. As a whole, the majority of NGSI districts and high schools visited had increases in student at- 
tendance. In addition, 8 of 12 districts had retention rate decreases that exceeded the state average de- 
creases (-0.8) and 5 of 8 districts had 2002 retention rates below the state average (16.9 percent). 

Algebra I End-of-Course exams did not compare as favorably with the state average gain. Of 12 com- 
parisons, 6 NGSI districts had larger gains than the state (-1-13.9 percentage points). Three participating 
districts — Amarillo, Galena Park, and Aldine — exceeded state benchmarks on both retention and algebra 
indicators. In general, student performance within individual high schools varied across districts with 
multiple campuses. 

Based on these data, it is impossible to conclude that NGSI was a success or a failure. In particular, com- 
parisons with state averages are between dissimilar groups. Systematic differences almost certainly exist 
between NGSI students and state comparison groups. NGSI students were selected for program participa- 
tion based on their academic needs. Students in the state comparison group were not. Even comparisons 
between visited campuses and district NGSI averages are suspect. Systematic differences may exist be- 
tween NGSI students from campus to campus within a district. Thus, any observed changes may be due to 
the NGSI program (or the program at a specific campus), or the changes may be due to preexisting aca- 
demic and motivational differences between the comparison groups and NGSI students. Thus, systematic 
differences make it difficult to prove whether NGSI was effective or not. 
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Table 5.1. NGSI Outcome Variables for Ninth Graders 


District/Recipient 

Attendance 

Rate 

2001-02 

Change 
1999-00 to 
2001-02 

Retention 

Rate 

2001-02 

Change 
1999-00 to 
2001-02" 

Algebra 

EOC 

2001-02 

Change 
1999-00 to 
2001-02” 

Crockett High School 

94.2 

H-0.1 

8.6 

-11.3 

16.8 

-8.2 

Los Fresnos High School 

94.3 

h-2.3 

18.8 

+0.8 

38.9 

-2.6 

Marshall High School 

95.8 

h-2.3 

15.8 

-7.0 

28.1 

+9.0 

San-Felipe-Del Rio CISD 

93.7 

-1.5 

9.9 

-2.0 

31.5 

+2.5 

Freshman School 

95.0 

h-4.5 

OO 

CO 

-1.0 

29.7 

+1.1 

High School 

95.2 

+15.8 

33.3 

-2.6 

4.6 

-0.5 

Amarillo ISD 

91.8 

+2.0 

12.8 

-2.2 

58.0 

+23.9 

Caprock High School 

91.0 

+1.9 

10.5 

-6.5 

33.1 

+24.5 

Beaumont ISD 

91.0 

+0.1 

26.4 

-2.9 

41.8 


Ozen High School 

92.8 

+3.1 

20.3 

-5.0 

10.0 


Galena Park ISD 

93.5 

-0.1 

12.8 

-3.4 

57.8 


North Shore High School 

93.8 

+1.6 

14.0 

+0.7 

48.2 


Aldine ISD 

93.8 

+0.5 

15.0 

-6.6 

74.2 


Nimiz Ninth Grade 

95.0 

+3.3 

11.7 

-6.8 

86.1 

+13.0 

Nimiz High School 

90.0 

+4.7 

35.8 

+10.1 

37.5 

+7.5 

Fort Worth ISD 

89.7 

-0.2 

26.8 

+0.8 

44.5 

+21.1 

Carter Riverside 

91.5 

+1.5 

17.5 

-12.9 

28.4 

+16.1 

Tremble Technical 

93.6 

+5.6 

5.7 

-2.8 

50.9 

+42.9 

San Antonio ISD 

91.5 

+0.1 

17.6 

-3.5 

46.4 

+12.6 

Lanier High School 

91.2 

+3.3 

21.1 

-3.9 

38.2 

+24.8 

Jefferson High School 

92.1 

+2.5 

14.5 

-3.7 

51.2 

+19.3 

Ysleta ISD 

93.9 

0.0 

16.5 

0.0 

57.1 

+15.2 

Del Valle High School 

94.8 

+1.1 

16.9 

+2.9 

73.6 

+26.3 

Socorro ISD 

94.8 

+1.0 

20.8 

+3.2 

46.3 

+17.8 

Socorro High School 

94.7 

-0.1 

16.9 

-9.8 

40.3 

+ 17.1 

State Average 

- 

- 

16.9 

-0.8 

57.8 

+13.9 


Note. State attendance data for ninth graders are unavailable. Bold indicates district change is greater 
than state average. District and state Algebra I EOC exam averages includes all students taking the exam, 
primarily 8th and 9th graders. 


How does the high school context and educational environment support students 
who are at risk? 


Each grant program operates within the broader campus and school district as a whole — therefore, to bet- 
ter understand student performance, researchers examined not only the NGSI program but also the school 
context experienced by ninth graders at risk of failure. Areas of interest arose from a review of recent re- 
search and publications offering recommendations for improvements in the nation’s high schools. Topics 
relating more broadly to the high school context included standards and expectations, structure and or- 
ganization, opportunities for extra academic assistance, and guidance and counseling services. Research- 
ers also gathered information on the high school environment and the nature of teaching and learning. 
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Standards and Expectations 

> In nearly all high schools visited, the Recommended High School Program Is currently the de- 
fault curriculum. 

> Many districts have established more rigorous promotion standards to ensure that ninth 
graders are prepared for the Texas Assessment of Knowledge and Skills (TAKS). 

Since the 77th Texas Legislature made the Recommended High School Program the default curriculum 
for the senior class of 2008, researchers gathered information on progress toward the adoption of higher 
standards. Texas high schools undoubtedly are endorsing more rigorous academic standards. Ninth grad- 
ers in 11 of 12 districts visited initially are enrolled in the Recommended Program, and the Minimum 
Plan only is considered as a last option to facilitate graduation. The advent of statewide testing in ninth 
grade also has led high schools to toughen student promotion standards. Many high schools now require 
students to complete six credits rather than five to advance to tenth grade, and some require students to 
complete core-suhject area eourses as well. 

Even though higher academic standards provide a basis for high sehool improvement, unintended conse- 
quences exist. In particular, when at-risk students fail and must repeat courses to accrue needed credits, 
their educational options begin to narrow. Both educators and students report that some students must cut 
back on electives or extracurricular activities such as sports or fine arts to retake classes or dedicate ex- 
tended time to coursework. Thus, higher standards can have detrimental effects that may further disen- 
gage students from school and learning. The present challenge for high schools is to help ninth graders 
succeed in core-subject courses the first time enrolled. 

Structure and Organization 

> Although most high schools retain the traditional grades 9-12 structure, some have created 
smaller, more supportive units within the high school. 

> Scheduling approaches vary widely, but high schools appear to be shifting from block sched- 
ules (90-mlnute periods) to traditional, single-period schedules (50-mlnute periods). 

> A few high schools modified their schedules to give extended learning time to ninth graders 
considered at-risk of academic failure, primarily in algebra and English. 

Many proponents of high school reform believe the manner in which high schools organize and use time 
affects the quality of teaching and learning. Recent research has focused on the benefits of creating 
smaller schools or smaller units within large comprehensive high schools (Harvey & Housman, 2004; 
Vander Ark, 2004). 

Researchers for this study found a few high schools experimenting with organizational structures as a way 
to bolster student achievement. However, the majority of Texas high schools visited still have traditional 
grades 9-12 structures and large student enrollments. Restructuring in these schools more often involves 
changed course schedules, time allocations, or staffing arrangements rather than new school configura- 
tions. For example, many high schools are abandoning block scheduling and returning to the traditional, 
single-period daily schedule, with students attending 7 or 8 classes each day throughout the school year. 

Changes appear to be driven by a belief that teachers need daily contact with at-risk students to prepare 
them to succeed on the TAKS. Some also think that block schedules with 90-minute periods have not 
produced the active, meaningful learning experiences or student success originally envisioned. Others feel 
that struggling students cannot maintain their focus in 90-minute periods or cope with the alternate-day 
schedules. 

A few exceptions exist to the traditional high school grade configuration. Two of 12 districts studied cre- 
ated ninth-grade schools with students housed in a separate building near an affiliated senior high school. 
This configuration reportedly benefits ninth graders by easing crowding (about 800-900 students per 
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school), reducing discipline problems, and creating an environment that allows maximum attention to 
students’ academic and emotional needs. Large high schools in other districts were re-designed as 
schools-within-a-school to provide a more supportive environment within schools typically enrolling 
about 2,000 students. One district recently implemented vertical teams (teams of grades 9-12 students, 
teachers, and support staff); however, not enough meeting time for teachers affected the envisioned col- 
laboration. High schools in another district had greater success implementing horizontal teams (teams of 
ninth-grade teachers, students, and staff). 

Overall, educators who successfully reconfigured large high schools into smaller, supportive units cited 
benefits such as eased student transitions to high school, strengthened communication among teachers, 
individualized attention for students, and a greater focus on student needs. Despite positive perceptions, 
researchers warn that the evidenced benefits of small schools, such as higher achievement, may not nec- 
essarily be generalized to schools-within-a-school (Howley, 2002). In particular, one may not expect to 
see the same effects unless the school-within-a school concept is implemented exactly as designed. In 
summarizing the current status of small schools, Harvey and Housman (2004) report that, “While scien- 
tific evidence supporting the efficacy of small schools is not yet available, many practitioners find that 
interacting on a smaller scale makes it possible to reach and support all students in personalized ways”. 

Teaming and Collaboration 

> Teachers believe high schools have clear goals and priorities, much cooperative effort, and a 
strong focus on student achievement, but they are less positive about their involvement in 
decision making and the enforcement of rules for student behavior. 

> In many high schools where departments are organized by subject area, teachers report few 
interdisciplinary meetings or meetings with peers for instructional planning. 

> Smaller high school units (school-within-a-school, ninth-grade center) seemed to promote bet- 
ter teacher collaboration. 

Multiple sources of evidence suggest that high school teachers have limited opportunities for interdisci- 
plinary collaboration due to complex course schedules, a lack of shared conference periods, and the 
school organization into subject-area departments. Discussions in formal meetings, according to teachers, 
frequently center on student test scores, interim exams, TAKS content, and test preparation. 

Many high school teachers rely on informal interactions with other teachers before or after school, during 
lunch, or between classes to discuss student problems or instructional issues. Teachers also express con- 
cerns about their involvement in decision-making in the high school. Teachers’ limited role in developing 
NGSI grant proposals certainly substantiates their view. However, high school teachers working in a 
ninth-grade school or as part of a school-within-a school team with dedicated planning time report more 
discussions about student problems and needs and greater opportunities for collaboration and professional 
development. 

Extra Academic Assistance 

> All sites visited offer extra academic assistance to students considered at risk, but some take 
a more structured approach. 

> Although educators and student participants believe tutorials are helpful, most at-risk stu- 
dents do not attend unless they are required. 

> Barriers to participation in tutorials include transportation issues, lack of motivation, schedul- 
ing difficulties, after-school conflicts, and perceived benefits. 

While the NGSI offered one means of extra academic assistance for struggling students, many districts 
and campuses implemented other programs as well. All high schools offer tutorials for at-risk students, 
with tutoring typically scheduled before, during, or after school or on Saturdays. Academic assistance 
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frequently helps students prepare for the state assessment (TAKS), complete assignments, or make-up 
assignments or excessive ahsences. In about half of the districts visited, tutorials are arranged informally 
between students and teachers. 

Other districts take a more structured approach and require students who have failing grades or who fail 
benchmark assessments to attend. In general, although both educators and students believe tutorials are 
helpful, student participation is a major problem in all districts. Many at-risk ninth graders said they sel- 
dom or never attend tutorials unless they are required. Of those who do participate, most indicated that 
they do not attend on a regular basis. 

The challenges in providing tutorials for at-risk students outside of regular school hours are similar to 
those cited previously for extended-day and summer programs. Students who have the greatest need are 
least likely to participate. Although a number of legitimate factors impede participation (e.g., transporta- 
tion, jobs, family responsibilities), many educators attribute poor attendance to students’ lack of motiva- 
tion. One teacher voiced a commonly held opinion: students who attend are “the ones that want to learn.” 

Students’ perspectives offer insight into their motives. Some ninth graders do not view tutorials as oppor- 
tunities for real academic improvement, feeling that brief tutorials do not help them understand material 
that was incomprehensible in class. Instead, students more often viewed tutorials as a way to make-up 
failed assignments or remove zero grades on assignments due to absences. Further, the inability of stu- 
dents to see long-term consequences usually meant that they waited until after failing a grading period or 
course to seek assistance. Overall evidence suggests that, although helpful, extra academic assistance out- 
side of regular school hours will not be enough to help many at-risk students meet rigorous academic 
standards. Learning opportunities during the regular school day must be strengthened as well. 

Guidance and Counseling 

> Guidance and counseling services for students in at-risk situations are limited in many high 
schools by counselor-to-student ratios that exceed recommended standards. 

> Contact between at-risk ninth graders’ and counselors is limited primarily to the selection of 
courses or programs; older students are more likely to receive information about jobs and ca- 
reers, or how to improve academic work. 

> Ninth graders’ interactions with counselors on high school plans occur most often in groups 
rather than individually. 

> Most students in at-risk situations report limited contact with counselors regarding higher 
education and career options, but access varies across districts and schools. 

> Students in at-risk situations generally have lofty educational aspirations that tend to diminish 
as students grow older. 

The important link between student motivation to learn and school achievement is well established. Sub- 
stantial evidence also shows that the school context can affect students’ beliefs about their competence 
and control, values and goals, and consequently, academic engagement (Institute of Medicine & National 
Research Council, 2004). In the school setting, access to guidance and counseling can help at-risk stu- 
dents establish personal goals and see how their current efforts in school yield future educational and ca- 
reer benefits. In light of the importance of educational goal setting, the 78th Texas Legislature mandated 
the development of personal graduation plans for middle and high school students in at-risk situations. 

Texas high schools and counselors clearly are trying to provide services for at-risk students. Even so, the 
counselor-to-student ratios in high schools visited (ranging between 1:243 and 1:535) leave limited time 
for personal attention. More often, counselor support focused on helping ninth graders select courses or 
high school programs. Planning, according to many, usually occurred in group sessions during spring vis- 
its to middle schools, with counselors helping students pre -register for high school, learn about course 
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requirements, identify career goals, and complete a coursework plan. When asked specifically about their 
high school plan, however, most ninth graders answered vaguely. 

Information on jobs, careers, and higher education is conveyed through various means, such as counsel- 
ors, career counselors. Career Connections courses, or the GEAR UP program. Despite efforts, most 
ninth graders said they had not discussed careers or educational opportunities with their counselor. Still, 
many students seemed interested and excited about the possibility of post-secondary education and ca- 
reers, almost all aspired to attend college or a vocational school, and many expected to graduate. 

Access to counseling services for at-risk students increases with age. Older students, who are more likely 
to fail and repeat ninth-grade courses, were more likely to report contact with counselors to get informa- 
tion on jobs or careers, for academic improvement, and to discuss things studied in class. Unfortunately, 
by the time counseling and guidance becomes more readily available, students’ lack of academic success 
appears to have diminished their hopes to participate and succeed in higher education. 

All of this speaks to the need for early intervention to help struggling students see possibilities for the fu- 
ture before they fail. Certainly, high school counselors cannot shoulder the full responsibility for guiding 
the many high school students who need help understanding their high school plan and how success in 
school relates to later opportunities in life. Some believe a promising strategy is to diffuse guidance and 
counseling responsibilities among school staff, including teachers. Trained professionals such as counsel- 
ors could serve as resources for staff and provide direct services for students and families with serious 
problems. Through this model, every student and family would have a school staff member as an adult 
advocate (National Research Council & National Institute of Medicine, 2004). 

Teachers and Teaching 
Qualifications and Assignments 

> Ninth-grade teachers are fairly experienced, but a substantial proportion comes to teaching 
through non-traditional certification. 

Survey results for ninth-grade teachers in the high schools visited revealed that 40 percent had joined the 
profession through alternative or post-bachelor certification programs. Although recruiting teachers 
through alternative means can be an effective way to fill critical vacancies, it also increases the need for a 
strong professional development program to build pedagogical knowledge among teachers who did not 
attend a standard teacher-preparation program. 

In several districts, educators raised concerns about whether the assignment of new and inexperienced 
teachers to ninth-grade courses undermines instructional quality and consistency — thus, compounding 
students’ learning problems. The inherent difficulties of teaching ninth graders (like large classes and 
immature students) appear to be contributing factors to class assignments. Several administrators said 
teachers view assignments to teach upper classmen and advanced classes as rewards for seniority. To ad- 
dress this issue, a few high schools report proactive efforts to assign more accomplished teachers for ninth 
graders. For example, some administrators assigned all language arts or mathematics teachers to at least 
one section of English I or Algebra I. 

Professional Development 

> High school teachers have access to professional development on a range of topics, with 
training delivered more often through workshops or a series of training sessions. 

Although professional development was not a strong focus for NGSI grants, teachers reportedly partici- 
pated in many workshops and training sessions relevant to ninth graders’ needs. Both administrators and 
teachers frequently noted that high school teachers are encouraged to use active learning strategies, dif- 
ferentiated instruction, and intellectually challenging activities. Although professional development deliv- 
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ery varied, teachers most often said they attended workshops or training sessions throughout the year. 
Educators seldom reported follow up to monitor implementation of instructional strategies. 

Perceptions of Effective instruction 

> High school teachers’ beliefs about teaching practices vary widely, with some advocating 
learner-centered approaches and others favoring traditional methods. 

> Students who are at risk say good teachers provide clear explanations, encourage active and 
meaningful learning, make class interesting, establish personal relationships, use small-group 
activities, and offer individual help. 

As a whole, teachers and students express similar views on certain instructional practices that effectively 
promote learning (see Box 5.4). Both groups advocate active and meaningful learning experiences, varied 
(or interesting) instructional approaches, and positive interpersonal relationships. Interestingly, many of 
these qualities are consistent with research on engaging adolescent learners (e.g., Lambert & McComhs, 
2000). 

Still, important differences also emerge. Many at-risk student say they learn and rememher more from 
teachers who make the subject matter understandable by explaining step-by-step, simplifying, using dif- 
ferent terminology, and persisting until students understand. Students also are more likely than teachers to 
cite benefits gained by working with their peers in small groups and receiving individual assistance from 
the teacher. 


Box 5.4. Effective Instructional Practices 


Teacher Perceptions 

Student Perceptions 

• 

Use hands-on activities 

• Provide clear explanations 

• 

Provide relevant, real-life experiences 

• Encourage active and meaningful learning 

• 

Use varied instructional approaches 

• Make class interesting 

• 

Hold students accountable 

• Establish personal relationships 

• 

Provide constant reinforcement 

• Use small-group activities 

• 

Build personal relationships 

• Offer individual assistance 

Note. Order reflects most frequently cited teacher and student perceptions from high to low. 

Source: Focus groups involving 124 teachers and 202 students. 


Although high school teachers agree on some instructional practices, they differ on others. A number of 
teachers advocate learner-centered approaches, but others believe traditional, teacher-directed instruction 
works best. In particular, some teachers felt that activities such as small-group instruction are not success- 
ful with at-risk students who are largely unmotivated learners. In some cases, teachers question whether 
active learning strategies will prepare students who lack the basic skills to do well on the TAKS. Several 
teachers believe that holding students more accountable for attendance, homework, grades, and discipline 
is the key to improving learning outcomes. 

The most noteworthy aspect of teachers’ views on instructional practices is the difference between ex- 
pressed opinions (regarding the need for hands-on activities, relevant experiences, and varied instruc- 
tional approaches) and observed practices (mainly teacher-centered classrooms). 


130 




Teachers’ Classroom Practices 

> High school classrooms are organized most often for whole-class instruction followed by stu- 
dents working independently. Students seldom work collaboratively with peers. 

> Teachers spend the greatest proportion of class time providing whole-class instruction and 
monitoring students as they work independently on assignments. 

> Teachers seldom ask mentally challenging questions or questions that help at-risk students 
see the relevance of subject matter to their lives. 

> Since teachers have little access to technology in classrooms, it is seldom used to support 
instruction and learning. 

Researchers conducted observations in 81 core subject-area classrooms in high schools. Interestingly, 
comparisons between observational findings for this study and results from a landmark study conducted 
by Goodlad in the 1980s (A Place Called School, 1984) show that not much has changed in high schools. 
High school teachers tend to teach one way — primarily whole-class lectures. 

Most teachers in this study organized their class for whole-class instruction, with student desks commonly 
arranged in rows facing the teacher. Following teacher-led lectures, explanations, or demonstrations, 
teachers usually walked around the room monitoring students as they worked independently. Teachers 
sometimes stopped briefly to answer a question or assist a student having difficulty with an assignment, 
but they rarely provided sustained individualized instruction. 

During whole-group discussions, teachers mainly relied on brief question and answer exchanges to estab- 
lish students’ grasp of factual information. Questions posed by teachers seldom required students to ex- 
plain concepts in their own words or to justify their ideas verbally. Rarely were questions used to help 
students connect concepts being studied in the lesson to real world applications or to other subject areas. 
Only 1 in 10 teachers used technology for lessons; those who did made PowerPoint presentations or used 
visuals to support whole-class lessons. 

Instructional practices observed in high schools are inconsistent with current research on how students 
learn or with known methods for engaging students. Research shows that teachers must build on their stu- 
dents’ preexisting knowledge, provide opportunities for them to become good thinkers (e.g., notice pat- 
terns, generate arguments and explanations, and draw analogies), and help students organize information 
to facilitate retrieval and application in other contexts. Teachers also must help students examine their 
own thinking and monitor their own understanding (i.e., teach metacognition) (Bransford, Brown, & 
Cocking, 2003). 

Although no one universally accepted instructional method exists, learner-centered environments create 
opportunities for active, meaningful, relevant, and intellectually challenging experiences that promote 
student engagement and achievement. (National Research Council & Institute of Medicine, 2004). 

Students and Learning 
Opportunities to Learn 

> Students at risk spend the greatest part of their time listening to teacher presentations or in- 
dependently completing short-answer activities or worksheets. 

> Students rarely use technology in classrooms to support content-area learning. 

The problem with teacher-centered classrooms is the effect on students. Students in the core-subject area 
classrooms observed spent nearly half of their time as listeners rather than active learners. Following 
teacher-led presentations, students usually worked alone to complete a worksheet or a short-answer exer- 
cise. Students seldom worked collaboratively with other students to share their thinking or discuss ideas. 
Further, students rarely used technology to support learning because computers in high school are usually 
located in labs rather than classrooms. 
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One-on-one teacher assistance typically was brief and usually in response to difficulty with an assign- 
ment. Thus, teachers had little time to understand student thinking processes or the knowledge and skills 
they brought to the lesson. In general, teachers expect little of at-risk students intellectually. Students sel- 
dom engaged in challenging activities promoting the kinds of thinking needed to meet state content stan- 
dards (e.g., analysis, synthesis, problem solving, application, elaborative communication) or to prepare 
them for more advanced coursework. 

Overall, observed practices and learning opportunities in high school classrooms raise questions about 
teachers’ understanding of students as learners, especially research-based conceptions. Observed practices 
also are inconsistent with learner-centered principles as advocated for Texas schools, assessed through the 
Professional Development and Appraisal System (PDAS), and shown to be effective for learners through 
a vast body of research (e.g., Bransford, Brown, & Cocking, 2000; Lambert & McCombs, 2000). Profes- 
sional development for high school teachers should focus on building an understanding of students as 
learners, as well as the implementation of content- specific instructional strategies that are linked to stu- 
dent achievement. 

Perceptions of Students as Learners 

> Educators believe ninth graders’ academic performance is affected by inadequate learning 
strategies and skills, immaturity and irresponsibility, lack of academic preparation, lack of mo- 
tivation, and poor attendance. 

Teachers’ perceptions of at-risk students as learners may partially explain their instructional approaches. 
Many teachers believe ninth graders come to high school with insufficient content knowledge and inade- 
quate learning strategies and skills to succeed academically. Thus, many teachers who believe students 
are disorganized, unmotivated, and lack self-discipline, think students will not learn unless the teacher 
maintains control of classroom activities. Disruptive students also may play a role in why some teachers 
do not use small-group activities. Nearly half of at-risk students indicate that disruptions by other students 
interfere with their learning. 

Evidence from various sources points to at-risk students’ disengagement from high school and learning. 
Poor attendance, lack of motivation, disruptive behavior, irresponsibility regarding homework and grades 
are all symptoms of larger problems. Findings throughout this study point to such issues as: 

• Boring and repetitive instruction in core subject-area classrooms that fails to engage students intellec- 
tually; 

• Limited use of technology in core-content classrooms to support engaged learning; 

• Expectations to attend after-school or Saturday tutorials when in-school time is not used to the great- 
est advantage; 

• Repeated course failure, which narrows educational choices and opportunities for enriched learning 
experiences; and 

• Poor access to counseling and advisement to help students set goals and see how current investments 
in learning yield future benefits. 

Although high schools cannot control all of the factors that influence engaged learning, high school edu- 
cators more often attribute the poor performance of at-risk students to socioeconomic and personal defi- 
ciencies or to inadequate preparation in middle schools. In contrast, the high school context and class- 
room experiences are seldom mentioned as important influences on student engagement, motivation to 
learn, and achievement. 
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Transition from Middie-to-High-Schooi 

> Differences in schooi size and organization, grading systems, educationai phiiosophy, teacher 
characteristics, and academic expectations make the transition from middie-to-high-schooi 
difficuit for ninth graders. 

> inadequate academic preparation, increased freedom coupied with immaturity, home-iife 
situations, and apathy makes high schooi chaiienging for many ninth graders. 

Both educators and students offer insights into difficulties experienced hy ninth graders in transitioning to 

high school (see Box 5.5). 


Box 5.5. Ninth Graders’ Chaiienges in Transitioning to High Schooi 


Middie-to-High Schooi Differences 

• School size and organizational features 

• Grading and credit system 

• Educational philosophy 

• Teacher characteristics 

• Academic expectations 


Student-Reiated issues 

• Inadequate academic preparation 

• Increased freedom coupled with immaturity 

• Home life situation and poverty 

• Apathy/lack of effort 


Note. Order reflects most frequently cited challenges from high to low. 

Source: Interviews involving high school administrators and directors (n=47) and focus groups with teachers 
(n=124) and students (n=202). 


The redesign of middle schools into smaller and more supportive learning environments (e.g., Turning 
Points, 2000) has made the passage from middle schools to large, impersonal high schools even more dif- 
ficult for students. Middle schools and high schools differ in size and structure, teachers have different 
instructional styles and attitudes, and grading systems also are vastly different. 

In general, high school administrators and teachers expect ninth graders to arrive with near-grade level 
content knowledge, adequate learning strategies, and the skills to work independently. Educators also ex- 
pect students to listen and learn from lectures, and to take responsihility for completing homework outside 
of class. Students who do not meet expectations have difficulty doing well in high school. Overall, com- 
munication between middle- and high-school educators should he strengthened to resolve misunderstand- 
ings arising from the juncture of two very different educational philosophies. Organizational and instruc- 
tional inconsistencies make the middle-to-high school transition difficult, especially for students in at-risk 
situations. Although vertical teaming has been touted as one way to achieve greater cohesion between 
school levels, little evidence emerged to suggest that strong lines of communication exist. Although it is 
up to each district to determine how to address issues that affect at-risk students, recommendations on 
high school reform offered by school administrators, researchers, policymakers, and the business commu- 
nity are worthy of consideration (e.g., Harvey & Houseman, 2004; NASSP, 2004). 


What are the implications for grant awards and management? 


Grant recipients generally praised TEA’s facilitation of the NGSI grant process. Recommendations con- 
cerning grant management typically related to the timing of grant awards and funding. Many grantees 
appreciated TEA efforts to streamline the evaluation process in later grant terms. In a few instances, edu- 
cators found the guidelines for allowable fund uses confusing, bindings to follow relate more specifically 
to overall improvement of grant development, implementation, monitoring, and sustainability. 
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Grant Development 

> Grant applications should put greater emphasis on identifying problems, determining the root 
causes, and articulating how the project will alleviate those problems. 

While most sites made use of student achievement, attendance, and retention data as outcome measures 
for grant development, most did not have well-developed processes for assessing school needs in a sys- 
tematic way. Specifically, reporting on trends in student achievement alone does not help to identify fac- 
tors that facilitate or hamper student learning. Grant developers from schools, with the assistance of dis- 
tricts, should consult with various stakeholders (especially teachers), examine various forms of data, and 
attempt to sort out root causes for difficulties. Once causes are identified, attention can he turned to ad- 
dressing them through the grant. Improving teaching and learning in schools might he accomplished 
more effectively if schools choose from rigorously researched and well-documented reform designs that 
provide networks of support for implementation (Slavin & Fashola, 1998). 

Grant programs should also have a clear focus, with a clearly organized explanation of how program 
components connect to interim and long-term outcomes. Individual NGSI award components often were 
implemented as discrete, disconnected activities. Schools and districts should he encouraged to think in 
terms of cause and effect with each component serving a unique and critical role in the overall program. 
This means ensuring that stakeholders understand the improvement effort, adequate initial and ongoing 
planning occur, and dedicated oversight for the award exists. 

> Grant applications should be informed by the thinking of various stakeholders. 

NGSI grant development primarily involved campus and district administrators. While administrative 
perspectives are critical, input from faculty, staff, and even parents and students can lead to a hetter- 
informed set of solutions, and to increased huy-in. 

> Grant programs for students at risk should be aligned with curricular and learning expecta- 
tions in regular classrooms. 

Several NGSI schools established arguably separate or dual curricula for at-risk students. If the purpose of 
providing services to students in at-risk situations is to move them to a new status of “at promise,” those 
students need access to the same curricula and high level of learning expectations that others receive. 
Several studies have demonstrated the harmful effects of tracking for low-performing students (Oakes, 
1985; Wheelock, 1992). 

In developing guidelines for grants, policymakers and agency staff should consider how allowable activi- 
ties influence the theory and pedagogy behind student learning experiences. As an example, the language 
in the Request for Application, Ninth Grade Success Initiative (November, 1999) says, “Grant funds from 
the NGSI may be used to (a) create new programs, (b) enhance existing programs, or (c) expand existing 
programs” (p. 6). 

While this is relatively general language, references to “programs” suggest that the solution to the educa- 
tional needs of ninth graders lies in some “magical” program rather than more broad-based school im- 
provement. For example, many districts responded to the proposal by purchasing self-paced computer- 
assisted programs for credit recovery. Credit recovery programs (both computer- and classroom-based) 
often created a separate set of learning experiences and expectations for at-risk students, tracking strug- 
gling students into classes with other low achievers. 

> Grants aimed at improving learning and academic performance of at-risk students must in- 
clude substantial investments in professional development, especially for classroom teach- 
ers. 

Most of the schools studied did not focus on professional development. Without guidance and informa- 
tion, few educators can effectively improve their schools and student learning. In particular, school per- 
sonnel need to have access to learning strategies appropriate to the intended goals of their improvement 
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efforts (NSDC, 2001). Educators also must be able to apply knowledge about human learning and change, 
which only can happen with a deep content knowledge of research-based instructional strategies. For in- 
stance, it appears that high school personnel are unfamiliar with strategies to integrate technology fully 
into the curriculum. 

Teachers also need content-specific professional development and ongoing support to improve instruction 
and learning in core-subject area classrooms, especially algebra. At the same time, educators must not 
operate under the assumption that all staff development leads to positive outcomes or that more staff de- 
velopment is better (Guskey, 1998). Currently, much is known about the kind of professional develop- 
ment that changes teacher practice, which can serve as a guide for grant development (Hawley & Valli, 
1999; Wisconsin Center for Education Research, 2003). 

> Grant applicants should have access to research-based information on effective instruction 
and school improvement. 

In many cases, it appears that school staff members are under-prepared to plan effective school improve- 
ment. Grant developers need to access the abundant research on educational change and reform strategies. 
According to Moffett (2000), “We know enough to act,” and “we cannot afford to ignore the research.” 
Currently, a wealth of information is available to guide reforms that support student learning and school 
improvement. The state also may consider sharing evaluations of programs and improvement initiatives to 
help guide decisions about applicability to certain contexts. 

Grant Implementation 

> Grants should require or strongly encourage the addition of dedicated program leaders. 

Schools with dedicated program management at both the district and campus level appeared to have the 
greatest success implementing and continuing their grants. This was especially true in larger districts. Full 
implementation of any grant depends on consistent leadership at the school level. Principals, who often 
are designated campus leaders, frequently have too many responsibilities to provide close oversight for 
grants. Eikewise, teachers who have no release time cannot oversee grant activities adequately. When 
large districts receive grants involving multiple campuses, dedicated oversight is needed both at the cen- 
tral administrative and campus levels to ensure consistent communication. In general, districts that re- 
ceive grants should use a combination of local and grant resources to ensure strong support for implemen- 
tation. 

> Major program changes made during the grant should require TEA approval. 

Several schools made substantial changes to their initiatives during implementation. In some cases, entire 
components were dropped. While mid-course adjustments in school improvement efforts often are needed 
to address changes in policy or demographic context, frequent changes do not allow time for impact, and 
they make measurement of success impossible. In many cases, the implemented NGSl program bore little 
resemblance to the program described in the grant proposal. TEA should require grant awardees to go 
through a formal review process for program modifications. 

Grant Monitoring 

> Legislators should fund external evaluations at the same time that grant programs are ap- 
proved or reauthorized. 

Historically, many state-level evaluations of grant programs are conducted after programs have been im- 
plemented. Though findings are informative, this post-hoc approach precludes the use of more rigorous 
scientific methods for evaluations (i.e., experimental or quasi-experimental designs) that allow valid in- 
ferences on program effects. In the future, legislators should consider funding external evaluations as pro- 
grams are approved or reauthorized. This way, research organizations can provide unbiased information 
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on program effectiveness in order to guide agency and legislative decision-making on educational pro- 
grams. 

> Districts and campuses receiving grants must be heid accountabie for TEA reporting require- 
ments and impiementation fideiity. 

Several sites failed to meet TEA reporting requirements. Without accurate and timely data, TEA and 
evaluators cannot stay abreast of each award school’s implementation progress, and thus cannot attest to 
its effectiveness. Several sites also made substantial changes to their implementation plans, making com- 
parison of outcome to input difficult, if not impossible. NGSI program changes more often reflected ex- 
pediency and opinion rather than systematic decision-making about program effectiveness. With mid- 
course changes, it is difficult to gauge effectiveness or monitor change in outcomes. 

> Grant awardees should have access to external technical support, assistance, and formative 
evaluation. 

It appears that several awardees did not maintain alliances with potential external assistance providers. 
Assistance providers can help the school implement effective, research-based strategies and bridge the 
gaps among schools, districts, and the state. While expertise often is available within schools and districts, 
technical assistance by external providers broadens the pool of knowledge from which schools and dis- 
tricts can draw. 

Grant Sustainability 

> Districts should have a contingency plan to address changes in grant leadership. 

Staff and administrator turnover undermined consistent grant implementation and frequently led to pro- 
gram changes. Eurther, it appears that leadership and staffing changes may have had the greatest negative 
impact on the implementation and continuation of NGSI programs. When major grant staffing changes 
occur, districts should submit a revised plan to show how grant activities will be sustained under new pro- 
ject leaders. In particular, site-based decisions should not be allowed to override grant obligations and 
agreements without prior approval from the TEA. 

> Grant awardees should create a context that Increases the likelihood of program success. 

Broad-based input into grant planning and development, thorough program planning, campus administra- 
tive support, and teacher “buy in” all were associated with successful grant implementation. More wide- 
spread support for grant development and implementation will help to alleviate the void left when key 
project leaders leave a school or district. Each grant should create a web of support for implementation 
and sustainability. 
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Appendix A 

Theory of Change Model for the NGSI/TASI/OEYP Evaluation 



Theory of Change Model 

Ninth Grade Success Initiative (NGSI), Texas After School Initiative (TASI), Optional Extended Year Program (OEYP) 
Contextual Issues Program Elements Assumptions Intermediate Outeomes Long-Term Outeomes 
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Appendix B 

Estimating TASI and NGSI Student Participation 



Estimating TASI Student Participation 

1) The total number of students reported by each district for each term was calculated. 

2) Missing term data were replaced with a student estimate based on the following algorithm: 
a The average number of students per term reported by each district was calculated by 

adding all term data and dividing by the number of reported terms, 
b Missing data was replaced with a weighted district term average, 
c The weight was calculated based on information submitted by those districts reporting 
all seven terms (18 districts). A proportion of students by term was calculated. The 
average number of students per term was 5,701. The proportion of students in Term 1 to 
this average was .64 - indicating that fewer students were served in Term 1 compared to 
the average. A similar proportion was calculated for each term ( T1 - .64, T2 - .97, T3 - 
.78, T4 - 1.06, T5 - 1.11, T6 - 1.18, T7 - 1.26. 

d Weighted district average = (district student average for all reported terms) * (term 
proportion). For example; if district A had an average student term count of 200 and 
had data missing for terms 1, 4, and 7 the missing data was computed thus: 

• terml data = (200) (.64) = 128 students 

• term4 data = (200) (1.06) = 212 students 

• term7 data = (200) (1.26) = 252 students 

3) Student records from each term were matched to create one set of students for each year, 
leaving only those students with a valid student ID code in the dataset. Percent of students 
with valid ID's ranged from 89% to 97%. These data were then merged to calculate the 
number of unique students served each year. To extrapolate from known unique students to 
number if all data were submitted and all data were accurate, the following algorithm was 
used: 

a The number of known unique students was adjusted based on the match rate to include 
students without accurate data. For example, the number of unique students in year two 
(15,173) was divided by the average valid data rate for the two terms (.923+.930)/2 = 

.926 to calculate number of unique students if all data had been used. 

• 15,173/ .926 = 16,385 

b The number of unique students if all data were accurate was then multiplied by the 
proportional difference between reported and reported plus missing students. 

• term extrapolated / term reported = proportion of all to reported 

• TERM 2: 17,462 / 11,142 = 1.57 

• TERM 3: 16,230 / 11,207 = 1.45 

• TERM 2 (1.57) + TERM 3 (1.45) divided by 2 = 1.51 

YEAR 2 unique students with 100% match (16,385) multiplies by proportion (1.51) = 
extrapolated unique students (24,742) 
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Estimating NGSI Student Participation 

1) The total number of students reported by each district for each term was calculated. 

2) Missing term data was replaced with a student estimate based on the following algorithm: 
a The average number of students per term reported by each district was calculated by 

adding all term data and dividing by the number of reported terms, 
b Missing data was replaced with a weighted district term average, 
c The weight was calculated based on information submitted by those districts reporting 
all seven terms (41 districts). A proportion of students by term was calculated. The 
average number of students per term was 5,313. The proportion of students in Term 1 to 
this average was .75 - indicating that fewer students were served in Term 1 compared to 
the average. A similar proportion was calculated for each term ( T1 - .75, T2 - .49, T3 - 
1.12, T4 - 1.24, T5 - .55, T6 - 1.19, T7 - 1.38, T8 - .60, T9 - 1.30, TIO - 1.56, Til - .83. 

d Weighted district average = (district student average for all reported terms) * (term 

proportion). For example; if district A had an average student term count of 200 and 
had data missing for terms 1, 4, and 7 the missing data was computed thus: 

• terml data = (200) (.75) = 150 students 

• term4 data = (200) (1.24) = 248 students 

• term7 data = (200) (1.38) = 276 students 

3) Student records from each term were matched to create one set of students for each year, 
leaving only those students with a valid student id code in the dataset. Percent of students 
with valid ID's ranged from 89% to 97%. This data was then merged to calculate the 
number of unique students served each year. To extrapolate from known unique students to 
number if all data were submitted and all data was accurate the following algorithm was 
used: 

a The number of known unique students was adjusted based on the match rate to include 
students without accurate data. For example, the number of unique students in year two 
(78,640) was divided by the average valid data rate for the three terms (.961+ .912 + 
.926)/3 = .933 to calculate number of unique students if all data had been used. 

• 78,640/ .933 = 84,287 

b The number of unique students if all data were accurate was then multiplied by the 
proportional difference between reported and reported plus missing students. 

• term extrapolated / term reported = proportion of all to reported 

• TERM 3: 59580/ 47354 = 1.26 

• TERM 4: 70922 / 61951 = 1.14 

• TERM 5: 24144 / 18186 = 1.33 

• TERM 3 (1.26) + TERM 4 (1.14) + TERM 5 (1.33) divided by 3 = 1.24 

YEAR 2 unique students with 100% match (84,287) multiplies by proportion (1.24) = 
extrapolated unique students (104,516) 
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Appendix C 

TAAS Reading and Mathematics Passing Rates by Grade 



Table C.1. TAAS Reading and Mathematics Passing Rates, First-Time Students 


Cohort 
Test Category 

Reading 

Mathematics 

TASI 

Students 

Non-TASI 

Students 

Achieveme 
nt Gap 

TASI 

Students 

Non-TASI 

Students 

Achieveme 
nt Gap 

Cohort 1 
N=ll,578 

Pretest 

67.9% 

79.5% 

11.7 

71.5% 

82.7% 

11.2 

Posttest 

72.7% 

82.2% 

9.5 

76.0% 

84.5% 

8.5 

Diff. 



-2.2 



-2.7 

Cohort 2 
N=18,394 

Pretest 

74.2% 

82.9% 

8.7 

80.7% 

87.1% 

6.4 

Posttest 

80.4% 

85.8% 

5.5 

83.6% 

87.6% 

4.0 

Diff. 



-3.2 



-2.4 

Cohort 3 
N=26,134 

Prefesf 

80.2% 

85.5% 

5.3 

86.9% 

90.4% 

3.5 

Posffesf 

85.5% 

88.8% 

3.3 

88.1% 

90.4% 

2.3 

Diff. 



-2.0 



-1.2 

Cohort 4 
N=32,210 

Prefesf* 

- 

- 

- 

- 

- 

- 

Posffesf 

78.3% 

82.7% 

4.4 

60.6% 

67.6% 

7.0 


®The state first administered the TAKS assessment in 2002-03; thus, pretests are tmavailable for TAKS. 


Tabie C.2. TAAS Reading and Mathematics Passing Rates, Grade 6 


Cohort 
Test Category 

Reading 

Mathematics 

TASI 

Students 

Non-TASI 

Students 

Achieveme 
nt Gap 

TASI 

Students 

Non-TASI 

Students 

Achieveme 
nt Gap 

Cohort 1 
N=3,742 

Pretest 

71.8% 

81.5% 

9.6 

77.7% 

86.4% 

8.7 

Posttest 

72.8% 

80.5% 

7.7 

76.1% 

83.1% 

7.0 

Diff. 



-1.9 



-1.7 

Cohort 2 
N=5,710 

Prefesf 

76.6% 

87.6% 

11.0 

84.6% 

91.3% 

6.7 

Posffesf 

72.6% 

82.2% 

9.6 

81.5% 

87.7% 

6.2 

Diff. 



-1.4 



-0.5 

Cohort 3 
N=8,640 

Prefesf 

82.8% 

88.6% 

5.8 

91.2% 

94.2% 

3.0 

Posffesf 

79.6% 

85.3% 

5.7 

88.4% 

90.7% 

2.3 

Diff. 



-0.1 



-0.7 

Cohort 4 
N=32,210 

Prefesf* 

- 

- 

- 

- 

- 

- 

Posffesf 

73.0% 

80.0% 

7.0 

62.1% 

72.1% 

10.0 


®The state first administered the TAKS assessment in 2002-03; thus, pretests are tmavailable for TAKS. 
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Table C.3. TAAS Reading and Mathematics Passing Rates, Grade 7 


Cohort 
Test Category 

Reading 

Mathematics 

TASI 

Students 

Non- 

TASI 

Students 

Achievem 
ent Gap 

TASI 

Students 

Non- 

TASI 

Students 

Achievem 
ent Gap 

Cohort 1 
N=4,458 

Pretest 

64.0% 

79.5% 

15.5 

68.8% 

81.6% 

12.8 

Posttest 

68.5% 

80.3% 

11.8 

74.4% 

84.6% 

10.2 

Diff. 



-3.7 



-2.6 

Cohort 2 
N=6,942 

Pretest 

72.8% 

80.1% 

7.4 

77.8% 

84.1% 

6.3 

Posttest 

81.8% 

84.9% 

3.1 

82.1% 

85.4% 

3.3 

Diff. 



-4.3 



-3.0 

Cohort 3 
N=9,228 

Prefesf 

74.8% 

81.5% 

6.7 

84.8% 

88.6% 

3.8 

Posffesf 

85.5% 

88.5% 

3.0 

86.8% 

89.0% 

2.2 

Diff. 



-3.7 



-1.6 

Cohort 4 
N=32,210 

Prefesf* 

- 

- 

- 

- 

- 

- 

Posffesf 

79.5% 

83.0% 

3.5 

59.9% 

66.9% 

7.0 


®The state first administered the TAKS assessment in 2002-03; thus, pretests are tmavailable for TAKS. 


Table C.4. TAAS Reading and Mathematics Passing Rates, Grade 8 


Cohort 
Test Category 

Reading 

Mathematics 

TASI 

Students 

Non- 

TASI 

Students 

Achievem 
ent Gap 

TASI 

Students 

Non- 

TASI 

Students 

Achievem 
ent Gap 

Cohort 1 
N=3,492 

Pretest 

67.2% 

76.3% 

9.2 

66.8% 

78.4% 

11.6 

Posttest 

77.4% 

85.9% 

8.5 

77.1% 

84.8% 

7.7 

Diff. 



-0.7 



-4.0 

Cohort 2 
N=5,930 

Prefesf 

72.5% 

80.4% 

7.9 

79.3% 

85.5% 

8.7 

Posffesf 

85.5% 

90.0% 

4.5 

87.0% 

89.6% 

5.5 

Diff. 



-3.4 



-3.2 

Cohort 3 
N=8,532 

Prefesf 

82.5% 

85.5% 

3.0 

83.6% 

87.6% 

4.0 

Posffesf 

91.3% 

91.9% 

0.7 

88.7% 

91.2% 

2.4 

Diff. 



-2.3 



-1.6 

Cohort 4 
N=32,210 

Prefesf* 

- 

- 

- 

- 

- 

- 

Posffesf 

81.9% 

84.2% 

2.3 

60.1% 

64.1% 

4.0 


®The state first administered the TAKS assessment in 2002-03; thus, pretests are tmavailable for TAKS. 
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Appendix D 


Case Study Site Selection Indicators 



TableD .1. Case Study Site Selection Indicators 
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Note. CAI=Computer-Assisted Instruction (i.e., PLATO Learning, NovaNET, CompassLearning, and Cognitive Tutor). Restructuring (i.e., teaming, 
school-within-a-school). Reconfiguration (i.e., reconfigured coursework, such as extended learning time, innovation instructional approach). 
^District has a separate school for ninth graders. 

I’Due to missing data, numbers under-represent total participants. 




